Restore VHD Image to Disk/Partition with Linux dd

Convert a VHD image from a native Windows backup to raw format using qemu-img, and write it directly to a disk or partition with the Linux dd command

I’ve recently been evaluating native Windows Server Backup as an option for bare-metal backup and recovery for our remaining physical servers at work. The utility creates several XML files and a VHD image for each partition it backs up. It seems to work ok for the most part, but I ran into a problem when I came across a system that for some unknown reason had a 38MB boot partition with insufficient space to create a VSS snapshot, thus preventing the tool from properly backing up the partition. I’ve read all the articles about allocating storage on a separate partition to get VSS to behave, but I could never get it to function correctly.

This got me thinking… I have some personal trust issues with the reliability of Microsoft products to begin with and now I’m having these problems which are just reinforcing the fear of something going wrong during the restore process. This lead me to start researching restore options using the VHD files produced by the native Windows Server Backup.

Option 1 is to do a restore using a Windows installation CD and select the restore option. Option 2 is to mount the VHD and manually copy files. This option is really only good for individual file restores. Obviously this is not something you’d want to do for a bare metal restore. Option 3 is to restore the VHD image directly to disk. This is the option I was most interested in and it made sense to me that there would be a straightforward way of doing this as every other bare metal backup solution I’ve used had this sort of option. While searching for a tool to do a VHD to disk image I found “VHD2Disk”. Unfortunately this tool was designed to do just that… write a VHD to DISK. No option for writing to a partition. Feel free to correct me if I’m wrong, but I see no way of ever getting 2 partitions on a single disk with this tool which makes it useless for my purposes.

After finding that there were no tools to do a image to disk write I became curious to find out if there was a way to just use the dd command in linux. After all, I would have instinctively turned to dd if this were something I were doing with linux. dd does block by block copying and a block is a block regardless of which OS you’re using. I quickly learned this is not something that can be done directly with a VHD because they are not in raw format, but the qemu-img supports VHD format and has the ability to convert them to raw image format. Below you will find the detailed instructions of how to convert the VHD, and use dd to write your new image to disk or partition. I’ll also give some details about getting the system to boot if you’re like me in this case and you don’t have a good backup of the boot partition.

The backup directory created by Windows Server Backup will look like this. I’ve highlighted the “interesting files” that I’ll mention throughout the article. The one ending in “Components.xml” has useful information about the disk partition layout that can come in handy when recreating partitions on your new disk. The .vhd file is the actual image data.

vhd image files

First thing we need to do is convert the VHD image to raw format. To do so, you’ll need access to a linux environment and use the qemu-img command. I’d recommend using either a Clonezilla or GParted liveCD as they both have all sorts of utilities pre-installed for disk imaging and partitioning. Boot the CD on the system you will be using for the restore image. When it finishes booting type the following commands to install qemu-img: (You may need to type sudo before each command if you’re not root. Keep this in mind for the remaining command as well.)

apt-get update
apt-get install qemu-utils -y

My backups are on a windows file share so I’ll use the following command to mount them to the /mnt directory:

mount -t cifs -o username=YOURUSERNAME,domain=YOURDOMAIN //HOSTNAME/PATH /mnt

You’ll be prompted for your password to mount the share. Be sure to replace YOURUSERNAME, YOURPASSWORD, HOSTNAME, and PATH with the information appropriate for your environment.

Next change into the directory containing the VHD file you need to convert: (the path used in this command may vary greatly depending on your environment)

cd /mnt/WindowsImageBackup/SERVERNAME/Backup\ 2016-08-05\ 133013/

Use the following command to convert the VHD image into raw format:

qemu-img convert -f vpc -O raw c9887432-6c68-11e0-a354-806e6f6e6963.vhd myserver.raw

You’ll need to repeat this command for any additional partitions you need to convert. Be sure to change the .vhd and .raw filenames to those appropriate for your environment. To be clear, the .vhd filename should be the one that exists in this directory like the highlighted file in the screenshot above, and the .raw filename can be whatever you want to name it.

You’ll notice a new file will be created that will reflect the full size of the partition for the data it contains. This is expected considering the nature of the raw image format.

-rwxr-xr-x 1 root root  668 Aug  5 13:51 BackupSpecs.xml
-rwxr-xr-x 1 root root  61G Aug  5 13:51 c9887432-6c68-11e0-a354-806e6f6e6963.vhd
-rwxr-xr-x 1 root root 149G Aug  5 21:39 myserver.raw

The conversion process can take a long time depending on the size of the partition. You can use the following command to output the status of the process: (I’ve noticed there is often some delay before the command writes to stdout)

root@debian# kill -SIGUSR1 `pidof qemu-img`
root@debian#     (22.03/100%)

Next you’ll need to create the 100MB boot partition (unless you’re restoring only a single partition and all others are fully intact) and any additional partitions the system originally had. I’ll assume you know how to do this, but you can use the output below for help if necessary. In the event that you don’t know the original partition layout, you can use the raw image size as a hint or the “Components.xml” file generated by Windows Server Backup in the backup directory for the server. With the values BytesPerSector, PartitionOffset, and PartitionLength contained in that file, you can re-create the exact partition table.

BytesPerSector / PartitionOffset = starting sector
(BytesPerSector / PartitionOffset) + (BytesPerSector / PartitionLength) = ending sector

fdisk /dev/sda

Welcome to fdisk (util-linux 2.27).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-335544319, default 2048):
Last sector, +sectors or +size{K,M,G,T,P} (2048-335544319, default 335544319): +100M

Created a new partition 1 of type 'Linux' and of size 100 MiB.

Command (m for help): t
Selected partition 1
Partition type (type L to list all types): 7
Changed type of partition 'Linux' to 'HPFS/NTFS/exFAT'.

Command (m for help): a
Selected partition 1
The bootable flag on partition 1 is enabled now.

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2):
First sector (206848-335544319, default 206848):
Last sector, +sectors or +size{K,M,G,T,P} (206848-335544319, default 335544319):

Created a new partition 2 of type 'Linux' and of size 159.9 GiB.

Command (m for help): t
Partition number (1,2, default 2): 2
Partition type (type L to list all types): 7

Changed type of partition 'Linux' to 'HPFS/NTFS/exFAT'.

Command (m for help): p
Disk /dev/sda: 160 GiB, 171798691840 bytes, 335544320 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5dcc5434

Device     Boot  Start       End   Sectors   Size Id Type
/dev/sda1  *      2048    206847    204800   100M  7 HPFS/NTFS/exFAT
/dev/sda2       206848 335544319 335337472 159.9G  7 HPFS/NTFS/exFAT

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

If you created the 100MB boot partition, format it as NTFS and the default “System Reserved” label:

mkfs.ntfs -f /dev/sda1 -L "System Reserved"

The VHDs always store the image as a partition within the image which means we have to get the offset to determine where the data actually begins in the raw image before we write it to disk. Use the following command:

fdisk -l myserver.raw

Disk myserver.raw: 148.1 GiB, 158967767040 bytes, 310483920 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device        Boot Start       End   Sectors   Size Id Type
myserver.raw1        128 310483071 310482944 148.1G  7 HPFS/NTFS/exFAT

The values “512” and “128” are what we need from this output. This tells us that the block size is 512 bytes and the partition starts at sector 128. Now we have all the information we need to give dd to write the image to our physical disk using this command:

dd if=myserver.raw bs=512 skip=128 of=/dev/sda2

You’ll need to repeat this command for any additional partitions you need to restore. You can use the following command to get the status of the dd process:

kill -SIGUSR1 `pidof dd`

369665+0 records in
369665+0 records out
189268480 bytes (189 MB) copied, 12.4497 s, 15.2 MB/s

When the dd command finishes writing the image to /dev/sda2, you should be able to mount the NTFS partition and view the files like so:

mkdir /sda2
mount -t ntfs-3g /dev/sda2 /sda2
ls -l /sda2
total 6267677
lrwxrwxrwx 2 root root         60 Jul 14  2009 Documents and Settings -> /sda2/Users
-rwxrwxrwx 1 root root 6327107584 Jul 31 11:03 pagefile.sys
drwxrwxrwx 1 root root       4096 Feb 17  2015 Patch Management
drwxrwxrwx 1 root root          0 Jul 14  2009 PerfLogs
drwxrwxrwx 1 root root       4096 Jun 17 07:44 ProgramData
drwxrwxrwx 1 root root       4096 Jul 21  2014 Program Files
drwxrwxrwx 1 root root       4096 Mar 22  2013 Program Files (x86)
drwxrwxrwx 1 root root          0 Apr 21  2011 Recovery
drwxrwxrwx 1 root root       8192 Apr 22 19:10 $Recycle.Bin
drwxrwxrwx 1 root root       4096 Aug  6 13:30 System Volume Information
drwxrwxrwx 1 root root       4096 May 30 14:16 Users
drwxrwxrwx 1 root root      24576 May 30 14:07 Windows

After you’ve restored all of your partitions, its time to reboot. Don’t forget to make sure the boot flag is on for your boot partition. If you didn’t have a copy of the boot partition, you’ll need to use the windows installation CD to repair the MBR. This usually involves a combination of the startup repair option available from the installation CD as well as some of the boot repair utilities that you can use from a command prompt on the windows installation CD. Like these:

bootrec /fixboot
bootrec /fixmbr
bootrec /rebuildbcd
cdromdriveletter:\boot\bootsect\bootrec /nt60 SYS /mbr

I recently found a cool new way of mounting the VHD image directly and imaging from the virtual block device instead of waiting for the qemu-img conversion and using up all of your precious storage for the VHD image you already have as well as a raw copy of the data. Below are the commands to enable the NBD kernel module with the right arguments, mount the VHD image as a virtual block device, and perform a dd copy to your physical disk. This is assuming you’ve already booted the liveCD, installed the qemu-utils, mounted the media containing your backups, and changed directly to the path containing the VHDs.

rmmod nbd
modprobe nbd max_part=16
qemu-nbd -c /dev/nbd0 c9887432-6c68-11e0-a354-806e6f6e6963.vhd
dd if=/dev/nbd0p1 of=/dev/sda2

You can see I’m using /dev/nbd0p1 as the source for the dd command. This is because as I mentioned earlier in the article, each VHD image contains a partition. nbd0p1 is referencing the first partition (the only partition) in the nbd0 virtual block device. Previously we had to specify the block size and offset with the dd command to specify where the partition started. Use the following command to remove the virtual block device for the VHD image.

qemu-nbd -d /dev/nbd0

If you have any questions or if you found this post useful, please leave a comment!

Delete User Profiles Remotely Windows XP/Vista/7/2008/2012

In my workplace, our helpdesk has a need for the ability to quickly and easily delete user profiles remotely. I did a little tinkering with wbemtest and found I could call the Delete() method on any of the WMI objects returned by the query “SELECT * FROM Win32_UserProfile.” It will properly delete the profile’s associated files and registry keys the same way that the windows native GUI tools do it. The problem with the native tool however, is that you need to be logged in to use it. It is fairly slow and clunky, and you can only select one profile at a time for deletion. This accounts for a lot of wasted time. So I took what I learned and created a little vbs script that made some WMI calls and deleted profiles. This worked great, but the help desk needs the ability to selectively choose which profiles get deleted through some form of user interface. I wanted the simplest possible solution that required no dependencies. (.NET, AutoIT DLL’s, etc). I found the best way to do that was to make an HTA application.

The first version of my profile cleanup HTA was very basic but served its purpose well. The problem was everything was done using synchronous WMI calls. I’ve recently been playing with a lot of Node.js to understand this whole “non-blocking IO” asynchronous programming methodology, and it got me wondering if I could do the same with this HTA application. It’s not difficult to find examples online for creating WMI queries and calling methods asynchronously, but getting them to play nice in the HTA application proved to be a challenge. At least for me : ).

One problem I had was certain things only worked using jScript, while other things only worked using VBscript. Fortunately I found a way to use both and reference functions in both languages from either language. The next problem I had was finding a way to reference the “WbemScripting.SWbemSink” object within HTA. The way I found to do it was by referencing the object by its class ID like so:

<object id="oSink" classid="clsid:75718C9A-F029-11D1-A1AC-00C04FB6C223"></object>

My first attempts at improving the UI was to make the function calls using the setTimeout javascript function but that didn’t seem to change anything. To prevent the windows from freezing I had to do everything asynchronously within WMI. I’m including links to both versions of the application. The old version should really only be used for educational purposes for developers interested in a before and after demonstration of asynchronous WMI vs standard synchronous WMI. The second version works quite well and is safe to use in production. Just be sure you don’t accidentally delete some important data in a user’s profile. Any comments, suggestions, improvements or questions are welcome!

Profile Cleanup Utility - Delete User Profiles

Download HTA Delete User Profiles Utility

Profile Cleanup

Profile Cleanup_V2

Two Factor Authentication with Freeradius for Horizon View

At work we were evaluating different options to enable two factor authentication for VMware Horizon View. They were all more than we were interested in paying and none had the ability to integrate with the communication platforms that we were interested in utilizing for delivering the PIN used as the “second factor”. Given that, my director gave me the opportunity to innovate and develop something custom.

Before we get started, you should know that I will not be providing a complete solution for two factor authentication with freeradius. My intention in this post is to demonstrate a working example of freeradius issuing an Access-Challenge response to a VMware View authentication request to achieve two factor authentication. Further development will be necessary to provide a full “solution”. (Integrating the freeradius perl module with LDAP or some other central authentication mechanism as well as deliver PINs and validate them.) If you have any questions in regards to how I achieved this, feel free to ask in the comments.

I had been looking for a good reason to play with freeradius and I finally had one. After some research within VMware’s documentation I knew I needed to learn how to get freeradius to send an “Access-Challenge” response.
“If the RADIUS server issues an access challenge, View Client displays a dialog box similar to the RSA SecurID prompt for the next token code.”

Unfortunately, getting freeradius to do this is not well documented, but here are a few links I used for my research:

I also read a few chapters from this book to get a better understanding of the configuration and inner workings of freeradius.

After all my research I used the code that comes with the freeradius perl module and modified the authenticate function like so:

sub authenticate {
        # For debugging purposes only
#       &log_request_attributes;
        if ($RAD_REQUEST{'State'} eq "0x6368616c6c656e6765") {
                if($RAD_REQUEST{'User-Password'} eq "1234") {
                        $RAD_REPLY{'Reply-Message'} = "Access granted";
                        return RLM_MODULE_OK;
                } else {
                        $RAD_REPLY{'Reply-Message'} = "Denied access by rlm_perl function";
                        return RLM_MODULE_REJECT;
        } else {
                if($RAD_REQUEST{'User-Name'} eq "testusernamehere" && $RAD_REQUEST{'User-Password'} eq "testpasswordhere") {
                        $RAD_REPLY{'State'} = "challenge";
                        $RAD_CHECK{'Response-Packet-Type'} = "Access-Challenge";
                        $RAD_REPLY{'Reply-Message'} = "Enter your PIN.";
                } else {
                        $RAD_REPLY{'Reply-Message'} = "Denied access by rlm_perl function";
                        return RLM_MODULE_REJECT;

The code above is extremely bare-bones and serves only as an example to use the perl module with freeradius to send an authenticator an Access-Challenge response to an authentication request. You will want to modify the “testusernamehere” and “testpasswordhere” strings to something more appropriate and optionally the “1234” test PIN. This code first authenticates a user by validating their username and password. If it is successful, an Access-Challenge response is sent to the authenticator and the “State” AVP (Attribute-Value Pair) is set to “challenge”. When the authenticator receives the Access-Challenge it prompts for a PIN. When the PIN is entered, the request is processed by the first block of code because the text value of the “State” AVP (challeng) now matches the hexadecimal string “0x6368616c6c656e6765” in the first if statement. This happens because in the previous request we set the State AVP to be equal to “challenge” which is the text equivalent to the hexadecimal string “0x6368616c6c656e6765”. The same User-Name is sent as used previously, but this time User-Password must match “1234”. Any other PIN will cause authentication to fail.

Here are screenshots of the Horizon View client authentication behavior using a freeradius server with this configuration.

two factor authentication vmware view first factor

two factor authentication vmware view second factor

Show multicast IGMP group memberships on Cisco IOS, Windows, and Linux

I’ve been doing a lot of playing with multicast lately and I always have to google for a while to find these commands. I figured it was time to throw a post together for a quick reference. Hopefully someone else can benefit from this too.

Below you can find the commands to determine whether a system or switch port is a member of a multicast group on Cisco IOS, windows and linux. Multicast uses IGMP to join these groups and there is no way to join a group manually. The operating system does it automatically when an application requests it so these commands can come in handy when you’re trying to figure out why you’re not seeing the multicast traffic that you’re expecting.

Cisco IOS:

show ip igmp snooping groups


netsh interface ip show joins


ip maddress show


netstat -ng

Windows screen recording with FFmpeg UScreenCapture and NGINX RTMP module

I recently came up with a unique and free way to do screen recording and broadcasting by leveraging a few unrelated, open source software components. The intention is not for brief screen captures, but to permanently record. Meaning, begin the recording on logon/unlock and stop at logoff/lock with the ability to monitor the session live, hear audio from the local microphone, and optionally activate the webcam and overlay it in a corner of the view.

Here’s a high-level overview of how everything will work:

  • NGINX is running with the RTMP module ready to receive RTMP AV streams and record them, making a new file every 5 minutes
  • FFmpeg launches at logon/unlock sending an RTMP stream to NGINX either locally or on a server remotely. It will use the UScreenCapture DirectShow filter and optionally connect to a local microphone and/or webcam.
  • During streaming, the session can be viewed live. FFplay, VLC, or flowplayer will works for this.
  • FFmpeg is killed at logoff/lock and the recording is stopped on NGINX.
  • Recordings can be viewed with ffplay or VLC.

Here’s what you’ll need to get it working:

I’m providing the NGINX build I found because it has the RTMP module compiled in, I’ve already put the stats.xsn file from the RTMP module in the html directory, and it already has the necessary configuration. It may not be the latest build out there, so feel free to use it as a reference for a better download you can probably find elsewhere.

To get everything in place, extract your ffmpeg download into C:\ffmpeg. This way the executable will be located at C:\ffmpeg\bin\ffmpeg.exe. Do a normal “next, next, finish” install of UScreenCapture. Finally, download the nginx zip and extract it to C:\nginx so that the executable is located at C:\nginx\nginx.exe. Feel free to install these components in alternative locations, but understand that you will need to modify the commands I provide accordingly.

Before we get ahead of ourselves, let’s make sure everything is working correctly. Start by opening a command prompt and typing “C:\ffmpeg\bin\ffmpeg.exe -list_devices true -f dshow -i dummy”. We need to make sure that the dshow filter “UScreenCapture” is listed in the output.

C:\ffmpeg\bin\ffmpeg.exe -list_devices true -f dshow -i dummy
ffmpeg version N-73266-g4aa0de6 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.9.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      54. 27.100 / 54. 27.100
  libavcodec     56. 45.101 / 56. 45.101
  libavformat    56. 40.100 / 56. 40.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 19.100 /  5. 19.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
[dshow @ 00000000032335c0] DirectShow video devices (some may be both video andaudio devices)
[dshow @ 00000000032335c0]  "USB Video Device"
[dshow @ 00000000032335c0]     Alternative name "@device_pnp_\\?\usb#vid_046d&pid_0825&mi_00#7&218d6046&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global"
[dshow @ 00000000032335c0]  "UScreenCapture"
[dshow @ 00000000032335c0]     Alternative name "@device_sw_{860BB310-5D01-11D0-BD3B-00A0C911CE86}\UScreenCapture"
[dshow @ 00000000032335c0]  "screen-capture-recorder"
[dshow @ 00000000032335c0]     Alternative name "@device_sw_{860BB310-5D01-11D0-BD3B-00A0C911CE86}\{4EA69364-2C8A-4AE6-A561-56E4B5044439}"
[dshow @ 00000000032335c0] DirectShow audio devices
[dshow @ 00000000032335c0]  "Microphone (USB Audio Device)"
[dshow @ 00000000032335c0]     Alternative name "@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\Microphone (USB Audio Device)"
[dshow @ 00000000032335c0]  "virtual-audio-capturer"
[dshow @ 00000000032335c0]     Alternative name "@device_sw_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\{8E146464-DB61-4309-AFA1-3578E927E935}"
[dshow @ 00000000032335c0]  "Microphone (Realtek High Defini"
[dshow @ 00000000032335c0]     Alternative name "@device_cm_{33D9A762-90C8-11D0-BD43-00A0C911CE86}\Microphone (Realtek High Defini"
dummy: Immediate exit requested

In the same command prompt, do the following:

cd C:\nginx

start "" nginx.exe

That should start nginx in the background and you should be able to browse to and see “Welcome to nginx for Windows!” I used port 81 in the configuration in C:\nginx\conf\nginx.conf to avoid conflict with other web servers that might be installed. If for some reason nginx isn’t working for you, check error.log located in C:\nginx\logs. If this is done in any sort of production configuration, I highly recommend compiling the latest build with the RTMP module on a linux server.

Now, from a command prompt, enter the command “C:\ffmpeg\bin\ffmpeg -analyzeduration 2147483647 -probesize 2147483647 -rtbufsize 1500M -f dshow -i video=”UScreenCapture” -c:v libx264 -vf “scale=trunc(iw/2)*2:trunc(ih/2)*2″ -crf 40 -profile:v baseline -x264opts level=31 -pix_fmt yuv420p -preset ultrafast -f flv rtmp://”. If you’d like you can use a streaming URL like rtmp:// I like to try and use something that will be unique if multiple streams are being broadcasted, but something that is also meaningful.

C:\ffmpeg\bin\ffmpeg -analyzeduration 2147483647 -probesize 2147483647 -rtbufsize 1500M -f dshow -i video="UScreenCapture" -c:v libx264 -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -crf 40 -profile:v baseline -x264opts level=31 -pix_fmt yuv420p -preset ultrafast -f flv rtmp://
ffmpeg version N-73266-g4aa0de6 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.9.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
  libavutil      54. 27.100 / 54. 27.100
  libavcodec     56. 45.101 / 56. 45.101
  libavformat    56. 40.100 / 56. 40.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 19.100 /  5. 19.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, dshow, from 'video=UScreenCapture':
  Duration: N/A, start: 860828.177000, bitrate: N/A
    Stream #0:0: Video: rawvideo, bgr24, 3200x1200, 10 tbr, 10000k tbn, 10 tbc
[libx264 @ 000000000322bee0] frame MB size (200x75) > level limit (3600)
[libx264 @ 000000000322bee0] MB rate (150000) > level limit (108000)
[libx264 @ 000000000322bee0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2AVX
[libx264 @ 000000000322bee0] profile Constrained Baseline, level 3.1
[libx264 @ 000000000322bee0] 264 - core 146 r2538 121396c - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=10 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=40.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, flv, to 'rtmp://':
    encoder         : Lavf56.40.100
    Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv420p, 3200x1200, q=-1--1, 10 fps, 1k tbn, 10 tbc
      encoder         : Lavc56.45.101 libx264
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
frame=   14 fps= 14 q=27.0 size=     134kB time=00:00:00.10 bitrate=10985.6kbits

If the stream is working properly, you should see some statistics at, and you should see recordings being generated within C:\nginx\recordings. Use VLC to play the recordings. To view the stream live with VLC click Media->Open Network Stream and enter the network URL “rtmp://”. Keep in mind that the username and computername here are case sensetive and should match exactly what is shown on the statistics page


Be patient as it can take some time for VLC to detect the video codec before it begins displaying. You can press “q” or Ctrl+c to stop the ffmpeg stream.

I did my best to tweak the command so that there is a good balance of quality and efficiency, but if you’d prefer higher quality video try changing the -crf parameter to a lower value like 23 or a slower -preset value like “fast”. A word of caution, the slower the preset you choose, the higher your CPU utilization will be. The “scale=trunc(iw/2)*2:trunc(ih/2)*2” part of the command is to avoid “not divisible by 2” errors when either your height or width stream resolution is an odd number. I ran into this in our VDI environment because you can resize the screen of the client to be any size you want and will frequently have this problem.

If video is all you need, at this point you can simply run the following vbs script using task scheduler with a logon and unlock event as the trigger:

Option Explicit

Dim WshShell

Set WshShell = CreateObject("Wscript.Shell")

WshShell.Run "C:\ffmpeg\bin\ffmpeg -analyzeduration 2147483647 -probesize 2147483647 -rtbufsize 1500M -f dshow -i video=""UScreenCapture"" -c:v libx264 -vf ""scale=trunc(iw/2)*2:trunc(ih/2)*2"" -crf 40 -profile:v baseline -x264opts level=31 -pix_fmt yuv420p -preset ultrafast -f flv rtmp://" & WshShell.ExpandEnvironmentStrings("%USERNAME%") & "-" & WshShell.ExpandEnvironmentStrings("%COMPUTERNAME%"), 0, False


To kill ffmpeg at logoff/lock, use task scheduler again with the appropriate triggers and run the command taskkill /f /im ffmpeg.exe.

When I first set out to get screen recording working for my purposes, I was originally attempting to save directly to an MP4 over a CIFS share, but I still had to kill the ffmpeg process because obviously we want in running in the background and there is no way to interact with the process to stop it gracefully. Terminating the process in this way would corrupt the MP4. With NGINX receiving the RTMP stream and handling all of the recordings independently of ffmpeg, you are able to kill the process without corrupting the video files.

Be sure to do some testing to make sure ffmpeg is terminating and launching correctly during the events you are using to trigger it. It is a good idea to set up an idle timeout/screensaver that locks your workstation and kills ffmpeg’s stream to avoid wasting storage on useless video.

I’ll try to post some more flexible/dynamic scripts later to demonstrate how to capture audio from the local microphone and overlay a webcam. If you have any input or questions, please comment below.

Bash script to migrate all KVM or Xen virtual machines to another host with virsh/libvirt

I’m working on setting up two fully redundant servers to host all sorts of services from the house. Most of the HA is automated via keepalived scripts, but I needed another one to automatically migrate all VMs from one host to another using libvirt. This is analogous to putting an ESXi host in “maintenance mode”. I thought I share the bash script I threw together.

First make sure you can successfully migrate manually then replace the $HOST variable with your target host and give it a shot. The script will first migrate all live VMs and then do an offline migration of all powered off VMs. Enjoy!



echo Migrating all VMs to $HOST

for VMS in `virsh list --name`; do echo Migrating VM $VMS live && virsh migrate --live --persistent --undefinesource $VMS qemu+ssh://lyasnode1/system; done

for VMS in `virsh list --all --name`; do echo Migrating VM $VMS offline && virsh migrate --offline --persistent --undefinesource $VMS qemu+ssh://lyasnode1/system; done

Convert all AVIs in your video library to MP4

I have a large video library and I’ve been on the look out for the best device to access all this media. It must support DLNA, not have cinavia, and obviously I’d like it to support as many audio and video codecs as possible. That eliminates most Sony products because they all seem to have Cinavia including PlayStation. I tried a chromecast and I won’t go into the details of how much I absolutely hated that useless piece of garbage. I still have a device running GoogleTV which is definitely my favorite, but unfortunately it has been discontinued by Google.

After much research I bought a Roku. I like it a lot, but it can be pretty picky about audio and video codecs. When videos have multiple audio streams whether it be DTS and stereo or multiple languages, the device will sometimes have no audio or play the wrong language. Fortunately, it is generally pretty simple to demux the streams and remap them in a way that the Roku will tolerate, but the device does not support AVI. This means if I want to keep the Roku around, I’ve either got to run Plex or some other transcoding capable DLNA server or convert all of my AVIs to H264 MP4s. I like to try to be as efficient as possible so which rules out transcoding a video every time you watch it, so I developed a little bash script to find all AVI files in my video library to MP4.

To run the script, you’ll need to have the perl-based “rename” utility installed as well as ffmpeg.

find /path/to/your/video/library/ -name "*.avi" -exec ffmpeg -i '{}' -c:v libx264 -crf 19 -preset slow -c:a libfaac -b:a 192k -ac 2 '{}'.mp4 \; -exec rename 's/.avi.mp4/.mp4/' "{}.mp4" \; -exec rm -f '{}' \;

Just change “/path/to/your/video/library/” to the real path to your video library and let the script do its thing. If you’d like to convert other video types, just change the search parameters “-name *.avi” to something that suits your needs. All videos will be re-encoded to H264 video, and 192k stereo AAC audio. It will then rename the file and delete the original file.

If anyone has any modifications or useful custom scripts you’d like to share, please leave them in the comments.


find /path/to/your/video/library/ -name "*.avi" -exec ffmpeg -i '{}' -c:v libx264 -crf 19 -preset slow -strict -2 -c:a aac -b:a 192k -ac 2 '{}'.mp4 \; -exec sh -c 'mv "$0.mp4" "${0%.avi}.mp4"' '{}' \; -exec rm -f '{}' \;

This one-liner doesn’t depend on a specific version of the rename utlilty. It also supports more versions of ffmpeg. The only flaw now is it only supports lowercase avi extension. Still working out the rename part of the script to handle that properly.

Enable Serial Console on CentOS/RHEL 7

Edit “/etc/sysconfig/grub”
Add to end of GRUB_CMD_LINELINUX, “console=ttyS0” Replace ttyS0 with your serial port.
Mine looks like this:

GRUB_CMDLINE_LINUX=" crashkernel=auto rhgb quiet console=ttyS0"

Run the following commands as root: Again replace ttyS0 with your serial port

stty -F /dev/ttyS0 speed 9600
grub2-mkconfig -o /boot/grub2/grub.cfg
systemctl start getty@ttyS0


Force Reboot Linux Remotely

I have a cubieboard set up at a friend’s house as a VPN and a backup target. I went to SSH into it the other day and found that almost every command I entered returned “Input/output error”. So I did the obvious and attempted a reboot, however both commands “reboot” and “init 6” returned “Segmentation fault” and did nothing. So I set out to find the most generic way to force reboot any linux distro regardless of systemd, kernel version or any other variable, and I found it! The method reminded me a lot of ALT + SysRq + REISUB only not as gentle considering the only signal it sends is a reboot. Maybe it can be modified to include the REIS & U, but here is the command I used in case someone else out there is in a hurry to get back online without concern for a safe shutdown!

echo 1 > /proc/sys/kernel/sysrq
echo b > /proc/sysrq-trigger

Use VLC to view RTSP stream from LaView LV-PB932F4 IP Camera

My co-worker scored an awesome deal for a new NVR system on 6 weatherproof H.264 1080P PoE IP cameras (LaView LV-PB932F4) with an 8 channel NVR (LaView LV-KN988P86A4). Now that I’ve gotten a chance to see how cool these cameras are, I’m wishing I’d taken advantage of the $599.99 sale.

I asked him to bring one of the cameras to work so I could check it out. The first thing I wanted to know was whether it supported the RTSP or protocol that would allow me grab the stream without proprietary software. Otherwise, the camera would be useless to me. I couldn’t find the RTSP URL in LaView’s documentation or anywhere on the web so I figured I’d blog about it in case someone else might benefit from the information.

After you’ve provided your camera with an IP using LaView’s utility (which at the time of this writing was available here IP address search software (SADP) on LaView’s download page) open VLC and click the Media option in the upper left toolbar:vlc_media

Enter the RTSP URL, including the username and password in the string like so. rtsp://admin:12345@x.x.x.x:554. The default credentials are used in the example. Username: admin, password: 12345 and the default RTMP port is 554. All of these settings can be modified by visiting the internal webserver of the camera and logging into the administrative interface.vlc_rtsp

Here is a screenshot of VLC while we were streaming to our family and friends the view of our new office. That’s my co-worker Mike on the left and me on the right. This camera has a beautiful picture!vlc_stream

This is pretty straightforward, but I wanted to make sure I posted the RTSP URL for these cameras so that I won’t forget and in case someone else out there is looking for it.

We went on from here to use ffmpeg to input the RTSP stream and segment an HLS (HTTP live stream) stream served by nginx through our reverse proxy. This allowed us to share a link with others to check out the camera live from outside the office. If anyone is interested in the details of the setup, please leave a comment and I’ll try to put together a how to.

UPDATE: I finally got one of these things for myself and I thought I’d share the RTSP URL for the DVR too (). The basic format is like so:


Most of it is self explanatory, but 101 is the HD stream for channel 1. 201 would be for channel 2, 301 for channel 3, etc. Also, you can type 102, 202, 302, etc for a low definition “thumbnail” stream for each of the channels. I was very excited to see this capability of the DVR because it opens up lots of possibilities with a little creativity and of course FFmpeg.