I have some old images of old Linux filesystems in flat file format. they can be used by Bochs, but I need to run them with Virtual Box. Virtual Box cannot use images in this format, so I need to convert these images from flat file to .vmdk file format. Is there any way to do this?
This question is related to
image
vmware
virtual-machine
virtualization
qemu
Maybe you should try using Starwind V2V Converter, you can get it from here - http://www.starwindsoftware.com/converter. It also supports IMG disk format and performs sector-by sector conversion between IMG, VMDK or VHD into and from any of them without making any changes to source image. This tool is free :)
Since the question mentions VirtualBox, this one works currently:
VBoxManage convertfromraw imagefile.dd vmdkname.vmdk --format VMDK
Run it without arguments for a few interesting details (notably the --variant flag):
VBoxManage convertfromraw
On windows, use https://github.com/Zapotek/raw2vmdk to convert raw files created by dd or winhex to vmdk. raw2vmdk v0.1.3.2 has a bug - once the vmdk file is created, edit the vmdk file and fix the path to the raw file (in my case instead of D:\Temp\flash_16gb.raw (created by winhex) the generated path was D:Tempflash_16gb.raw). Then, open it in a vmware virtual machine version 6.5-7 (5.1 was refusing to attach the vmdk harddrive). howgh!
Since the question mentions VirtualBox, this one works currently:
VBoxManage convertfromraw imagefile.dd vmdkname.vmdk --format VMDK
Run it without arguments for a few interesting details (notably the --variant flag):
VBoxManage convertfromraw
To answer TJJ: But is it also possible to do this without copying the whole file? So, just to somehow create an additional vmdk-metafile, that references the raw dd-image.
Yes, it's possible. Here's how to use a flat disk image in VirtualBox:
First you create an image with dd in the usual way:
dd bs=512 count=60000 if=/dev/zero of=usbdrv.img
Then you can create a file for VirtualBox that references this image:
VBoxManage internalcommands createrawvmdk -filename "usbdrv.vmdk" -rawdisk "usbdrv.img"
You can use this image in VirtualBox as is, but depending on the guest OS it might not be visible immediately. For example, I experimented on using this method with a Windows guest OS and I had to do the following to give it a drive letter:
You might want to access your files on Linux. First dismount it from the guest OS to be sure and remove it from the virtual machine. Now we need to create a virtual device that references the partition.
sfdisk -d usbdrv.img
Response:
label: dos
label-id: 0xd367a714
device: usbdrv.img
unit: sectors
usbdrv.img1 : start= 63, size= 48132, type=4
Take note of the start position of the partition: 63. In the command below I used loop4 because it was the first available loop device in my case.
sudo losetup -o $((63*512)) loop4 usbdrv.img
mkdir usbdrv
sudo mount /dev/loop4 usbdrv
ls usbdrv -l
Response:
total 0
-rwxr-xr-x. 1 root root 0 Apr 5 17:13 'Test file.txt'
Yay!
To answer TJJ: But is it also possible to do this without copying the whole file? So, just to somehow create an additional vmdk-metafile, that references the raw dd-image.
Yes, it's possible. Here's how to use a flat disk image in VirtualBox:
First you create an image with dd in the usual way:
dd bs=512 count=60000 if=/dev/zero of=usbdrv.img
Then you can create a file for VirtualBox that references this image:
VBoxManage internalcommands createrawvmdk -filename "usbdrv.vmdk" -rawdisk "usbdrv.img"
You can use this image in VirtualBox as is, but depending on the guest OS it might not be visible immediately. For example, I experimented on using this method with a Windows guest OS and I had to do the following to give it a drive letter:
You might want to access your files on Linux. First dismount it from the guest OS to be sure and remove it from the virtual machine. Now we need to create a virtual device that references the partition.
sfdisk -d usbdrv.img
Response:
label: dos
label-id: 0xd367a714
device: usbdrv.img
unit: sectors
usbdrv.img1 : start= 63, size= 48132, type=4
Take note of the start position of the partition: 63. In the command below I used loop4 because it was the first available loop device in my case.
sudo losetup -o $((63*512)) loop4 usbdrv.img
mkdir usbdrv
sudo mount /dev/loop4 usbdrv
ls usbdrv -l
Response:
total 0
-rwxr-xr-x. 1 root root 0 Apr 5 17:13 'Test file.txt'
Yay!
Just to give you an another option, you could use https://sourceforge.net/projects/dd2vmdk/ as well. dd2vmdk is a *nix-based program that allows you to mount raw disk images (created by dd, dcfldd, dc3dd, ftk imager, etc) by taking the raw image, analyzing the master boot record (physical sector 0), and getting specific information that is need to create a vmdk file.
Personally, imo Qemu and the Zapotek's raw2vmdk tools are the best overall options to convert dd to vmdks.
Disclosure: I am the author of this project.
krosenvold's answer inspired the following script which does the following:
the script is restartable and checks the existence of the intermediate files. It also uses pv and qemu-img -p to show the progress of each step.
In my environment 2 x Ubuntu 12.04 LTS the steps took:
#!/bin/bash
# get a dd disk dump and convert it to vmware
# see http://stackoverflow.com/questions/454899/how-to-convert-flat-raw-disk-image-to-vmdk-for-virtualbox-or-vmplayer
# Author: wf 2014-10-1919
#
# get a dd dump from the given host's given disk and create a compressed
# image at the given target
#
# 1: host e.g. somehost.somedomain
# 2: disk e.g. sda
# 3: target e.g. image.gz
#
# http://unix.stackexchange.com/questions/132797/how-to-use-ssh-to-make-a-dd-copy-of-disk-a-from-host-b-and-save-on-disk-b
getdump() {
local l_host="$1"
local l_disk="$2"
local l_target="$3"
echo "getting disk dump of $l_disk from $l_host"
ssh $l_host sudo fdisk -l | egrep "^/dev/$l_disk"
if [ $? -ne 0 ]
then
echo "device $l_disk does not exist on host $l_host" 1>&2
exit 1
else
if [ ! -f $l_target ]
then
ssh $l_host "sudo dd if=/dev/$disk bs=1M | gzip -1 -" | pv | dd of=$l_target
else
echo "$l_target already exists"
fi
fi
}
#
# optionally install command from package if it is not available yet
# 1: command
# 2: package
#
opt_install() {
l_command="$1"
l_package="$2"
echo "checking that $l_command from package $l_package is installed ..."
which $l_command
if [ $? -ne 0 ]
then
echo "installing $l_package to make $l_command available ..."
sudo apt-get install $l_package
fi
}
#
# convert the given image to vmware
# 1: the dd dump image
# 2: the vmware image file to convert to
#
vmware_convert() {
local l_ddimage="$1"
local l_vmwareimage="$2"
echo "converting dd image $l_image to vmware $l_vmwareimage"
# convert to VMware disk format showing progess
# see http://manpages.ubuntu.com/manpages/precise/man1/qemu-img.1.html
qemu-img convert -p -O vmdk "$l_ddimage" "$l_vmwareimage"
}
#
# show usage
#
usage() {
echo "usage: $0 host device"
echo " host: the host to get the disk dump from e.g. frodo.lotr.org"
echo " you need ssh and sudo privileges on that host"
echo "
echo " device: the disk to dump from e.g. sda"
echo ""
echo " examples:
echo " $0 frodo.lotr.org sda"
echo " $0 gandalf.lotr.org sdb"
echo ""
echo " the needed packages pv and qemu-utils will be installed if not available"
echo " you need local sudo rights for this to work"
exit 1
}
# check arguments
if [ $# -lt 2 ]
then
usage
fi
# get the command line parameters
host="$1"
disk="$2"
# calculate the names of the image files
ts=`date "+%Y-%m-%d"`
# prefix of all images
# .gz the zipped dd
# .dd the disk dump file
# .vmware - the vmware disk file
image="${host}_${disk}_image_$ts"
echo "$0 $host/$disk -> $image"
# first check/install necessary packages
opt_install qemu-img qemu-utils
opt_install pv pv
# check if dd files was already loaded
# we don't want to start this tedious process twice if avoidable
if [ ! -f $image.gz ]
then
getdump $host $disk $image.gz
else
echo "$image.gz already downloaded"
fi
# check if the dd file was already uncompressed
# we don't want to start this tedious process twice if avoidable
if [ ! -f $image.dd ]
then
echo "uncompressing $image.gz"
zcat $image.gz | pv -cN zcat > $image.dd
else
echo "image $image.dd already uncompressed"
fi
# check if the vmdk file was already converted
# we don't want to start this tedious process twice if avoidable
if [ ! -f $image.vmdk ]
then
vmware_convert $image.dd $image.vmdk
else
echo "vmware image $image.vmdk already converted"
fi
Maybe you should try using Starwind V2V Converter, you can get it from here - http://www.starwindsoftware.com/converter. It also supports IMG disk format and performs sector-by sector conversion between IMG, VMDK or VHD into and from any of them without making any changes to source image. This tool is free :)
On windows, use https://github.com/Zapotek/raw2vmdk to convert raw files created by dd or winhex to vmdk. raw2vmdk v0.1.3.2 has a bug - once the vmdk file is created, edit the vmdk file and fix the path to the raw file (in my case instead of D:\Temp\flash_16gb.raw (created by winhex) the generated path was D:Tempflash_16gb.raw). Then, open it in a vmware virtual machine version 6.5-7 (5.1 was refusing to attach the vmdk harddrive). howgh!
Just to give you an another option, you could use https://sourceforge.net/projects/dd2vmdk/ as well. dd2vmdk is a *nix-based program that allows you to mount raw disk images (created by dd, dcfldd, dc3dd, ftk imager, etc) by taking the raw image, analyzing the master boot record (physical sector 0), and getting specific information that is need to create a vmdk file.
Personally, imo Qemu and the Zapotek's raw2vmdk tools are the best overall options to convert dd to vmdks.
Disclosure: I am the author of this project.
krosenvold's answer inspired the following script which does the following:
the script is restartable and checks the existence of the intermediate files. It also uses pv and qemu-img -p to show the progress of each step.
In my environment 2 x Ubuntu 12.04 LTS the steps took:
#!/bin/bash
# get a dd disk dump and convert it to vmware
# see http://stackoverflow.com/questions/454899/how-to-convert-flat-raw-disk-image-to-vmdk-for-virtualbox-or-vmplayer
# Author: wf 2014-10-1919
#
# get a dd dump from the given host's given disk and create a compressed
# image at the given target
#
# 1: host e.g. somehost.somedomain
# 2: disk e.g. sda
# 3: target e.g. image.gz
#
# http://unix.stackexchange.com/questions/132797/how-to-use-ssh-to-make-a-dd-copy-of-disk-a-from-host-b-and-save-on-disk-b
getdump() {
local l_host="$1"
local l_disk="$2"
local l_target="$3"
echo "getting disk dump of $l_disk from $l_host"
ssh $l_host sudo fdisk -l | egrep "^/dev/$l_disk"
if [ $? -ne 0 ]
then
echo "device $l_disk does not exist on host $l_host" 1>&2
exit 1
else
if [ ! -f $l_target ]
then
ssh $l_host "sudo dd if=/dev/$disk bs=1M | gzip -1 -" | pv | dd of=$l_target
else
echo "$l_target already exists"
fi
fi
}
#
# optionally install command from package if it is not available yet
# 1: command
# 2: package
#
opt_install() {
l_command="$1"
l_package="$2"
echo "checking that $l_command from package $l_package is installed ..."
which $l_command
if [ $? -ne 0 ]
then
echo "installing $l_package to make $l_command available ..."
sudo apt-get install $l_package
fi
}
#
# convert the given image to vmware
# 1: the dd dump image
# 2: the vmware image file to convert to
#
vmware_convert() {
local l_ddimage="$1"
local l_vmwareimage="$2"
echo "converting dd image $l_image to vmware $l_vmwareimage"
# convert to VMware disk format showing progess
# see http://manpages.ubuntu.com/manpages/precise/man1/qemu-img.1.html
qemu-img convert -p -O vmdk "$l_ddimage" "$l_vmwareimage"
}
#
# show usage
#
usage() {
echo "usage: $0 host device"
echo " host: the host to get the disk dump from e.g. frodo.lotr.org"
echo " you need ssh and sudo privileges on that host"
echo "
echo " device: the disk to dump from e.g. sda"
echo ""
echo " examples:
echo " $0 frodo.lotr.org sda"
echo " $0 gandalf.lotr.org sdb"
echo ""
echo " the needed packages pv and qemu-utils will be installed if not available"
echo " you need local sudo rights for this to work"
exit 1
}
# check arguments
if [ $# -lt 2 ]
then
usage
fi
# get the command line parameters
host="$1"
disk="$2"
# calculate the names of the image files
ts=`date "+%Y-%m-%d"`
# prefix of all images
# .gz the zipped dd
# .dd the disk dump file
# .vmware - the vmware disk file
image="${host}_${disk}_image_$ts"
echo "$0 $host/$disk -> $image"
# first check/install necessary packages
opt_install qemu-img qemu-utils
opt_install pv pv
# check if dd files was already loaded
# we don't want to start this tedious process twice if avoidable
if [ ! -f $image.gz ]
then
getdump $host $disk $image.gz
else
echo "$image.gz already downloaded"
fi
# check if the dd file was already uncompressed
# we don't want to start this tedious process twice if avoidable
if [ ! -f $image.dd ]
then
echo "uncompressing $image.gz"
zcat $image.gz | pv -cN zcat > $image.dd
else
echo "image $image.dd already uncompressed"
fi
# check if the vmdk file was already converted
# we don't want to start this tedious process twice if avoidable
if [ ! -f $image.vmdk ]
then
vmware_convert $image.dd $image.vmdk
else
echo "vmware image $image.vmdk already converted"
fi
Source: Stackoverflow.com