Technical Note

Designing Backup Solutions for VMware vSphere

VMware vStorage APIs for Data Protection
Thisdocumentintroducescodedeveloperstotheconceptsandproceduresnecessarytocreatebackupand restoresoftwareforvirtualmachinesandVMwarevSphere(formerlyVMwareInfrastructure,orVI).You shouldbefamiliarwithprogrammingconcepts,practices,andtechniques.Youshouldalsobefamiliarwith theseconcepts:virtualmachine,snapshot,andvSphere(ESXandvCenter).Thisdocumentdoesnotduplicate informationavailableelsewhere.Whennecessary,itreferstootherdocuments. YoushouldbefamiliarwithnavigatingtheWebbasedVMwarevSphereAPIReferenceDocumentation,andread partsoftheVMwarevSphereWebServicesSDKProgrammingGuide,especiallytheintroductorychapters.Both areavailableathttp://www.vmware.com/support/developer/vcsdk.Thisbackgroundreadingcoversmanyof thebasicconceptsthatareimportantforvSpheredeployments.Thisdocumentisdividedintofoursections:

ConceptualOverviewonpage 1 HighLevelImplementationonpage 2 LowLevelProceduresonpage 8 ChangedBlockTrackingonVirtualDisksonpage 21 IncrementalRestoreofBackupDataonpage 24

Ifyouarelookingforahighleveloverviewofbackupandrestore,readthefirstsection.Ifyouaredesigning yourtoplevelprogramstructure,alsoreadthesecondsection.Implementersoflowlevelcodeshouldalso readthethirdsection.Eachsectionassumesthatyouhaveabsorbedknowledgefromtheprecedingsection.

Conceptual Overview
Thissectionsummarizesthebackupprocessandtherestoreprocess.

The Backup Process

Becausesnapshotsareaviewofavirtualmachinecorrespondingtoacertainpointintime,theyallowfora quickandcleanbackupoperation.Thefollowingstepsdescribeatypicalbackupworkflow: 1 2 3 4 5 Contacttheservermachinecontainingthetargetvirtualmachine. Commandthatservertoproduceasnapshotofthetargetvirtualmachine. Usetheservertogainaccesstothevirtualdisk(s)andfilesinthesnapshot. Capturethevirtualdiskdataandvirtualmachineconfigurationinformation(vim.vm.ConfigInfo). Commandtheservertodestroythebackupsnapshot.

Asideeffectofsteponeisbeingabletodeterminethearrangementanddescriptionofvirtualmachinesonthe server.Thisinformationisusefulfordeterminingthetargetvirtualmachine.
Copyright 2009 VMware, Inc. All rights reserved. 1

Designing Backup Solutions for VMware vSphere

The Restore Process

Youcanchooseoneoftworestorescenarios: To bring an existing virtual machine to a particular state 1 2 3 Contacttheserverandcommandittohaltandpoweroffthetargetvirtualmachine. Usetheservertogainaccesstothevirtualdisk(s). Transfertheimage(s)ofthedisk(s)fromthebackupprogram.

To completely re-create a virtual machine 1 2 3 Contacttheserver. Commandtheservertocreateanewvirtualmachineanditsvirtualdisksusingtheconfiguration informationinstep4ofthebackupprocessabove. Transferthevirtualdiskdatatothenewlycreatedvirtualdisk(s).Thisincludesthevirtualdiskformatting information,soitisunnecessarytobuildanykindoffilesystemonthevirtualdisk(s).

Summary of Requirements
Abackupprogrammustbeableto:

ContactaVMwarehost,displayinformationaboutthevirtualmachinesthatitcontrols,andmanipulate thosevirtualmachines. Forbackup,instructeachvirtualmachinetocreateatemporarysnapshot,andtransfersnapshotdatato thebackupapplication. Forrestore,instructthehosttohaltorrecreateatargetvirtualmachine,andrestorethedataforthat virtualmachinefromthebackupapplication.

High Level Implementation

Thissectionintroduces,atahighlevel,theVMwaresoftwareabstractionsneededtoaccomplishthesteps outlinedintheprevioussection.Theseabstractionsfallintothefollowingcategories:

Contactingtheserver Extractinginformationfromtheserver Causingtheservertoperformactions Givinginstancespecificcommandsandcontrol Transferringdata

Exceptforthelast,datatransfer,alltheabovecategoriesaresupportedbytheVMwarevSphereSDK,which containsawidevarietyofmanagementandcontrolinterfaces.

Communicating With the Server

InatypicalVMwarevSpheredeploymentthatconsistsofmultipleESXhosts,aninstanceofthevCenter typicallymanagestheESXhosts.ItisthereforerecommendedthatapplicationscommunicatewiththevCenter ratherthanwiththeindividualESXhosts. VMwarevCenterprovideslocationtransparencytotheuserofthevSphereSDK.VMwarevCentertracks virtualmachinesastheymove(throughVMotion)fromoneESXhosttoanother,anditdirectsvSphereSDK operationstotheappropriateESXhostthatcurrentlyhoststhevirtualmachine. ThehandlingofavCenteroranindividualESXhostisessentiallyequivalentwhenusingthevSphereSDK. ThuswhenvCenterispresent,thereisnoneedtodirectlycontactindividualESXhosts.Theremainderofthis documentusesthetermvSpheretoindicateeitheravCenteroranESXhost.

Copyright 2009 VMware, Inc. All rights reserved.

Designing Backup Solutions for VMware vSphere

UsingvCenterorthevSphereAPI,itisevenpossibletoidentifyanyVirtualApp(vApp)thatisinstalledand whatvirtualmachinesareassociatedwiththeit.ThismakesitpossibletobackupthevAppasaunit. ToreducetheresourcesusedbythevSphere,itisgenerallyrecommendedthatthenumberofconnections(or Sessions)toitbeminimized.Asaresult,itisinthebestinterestsofanyprogramthatcommunicateswiththe vSpheretocreateoneSessionandshareitwithallelementsoftheprogramthatneedtoexchangeinformation withtheserver.Thismeansthatifyourprogramsupportsmultiplethreads,thenyourprogramneedsto multiplextheuseoftheconnectionobjectthroughtheuseofaccesscontrollocks(mutexandthelike). ItisalsoimportanttonotethatallvSphereSDKoperationsproceedfromaninstanceofaSessionobjectthat yourapplicationrequestsafterloggingintothevSphere.Yourapplicationmightalsocreateobjectsthrough thevSphereSDKthatareSessionspecificandthereforewouldnotbeknowntoportionsofyourapplication thatareusingadifferentSession.

Information Containers as Managed Objects

VMwaredocumentationintroducesyoutotheconceptoftheManagedObjectanditshandle(calledaMoRef forManagedObjectReference).YoumightbetemptedtogetconfigurationandstatusinformationofManaged Objectsusingthepiecemealapproach.Thishastheseveredisadvantageofcreatingalotofchatteroverthe connectiontotheserver,anditisveryslow.Amechanismhasbeencreatedtoprovidethisinformation efficiently.ThismechanismisthePropertyCollector,discussedinPropertyCollectorUseonpage 4.

More About Managed Objects

WhenyoureadthedocumentationontheVMwarevSphereObjectModel,youareintroducedtoalarge numberofManagedObjects.ThereareonlyfivebasictypesofManagedObjectsthatdescribetheorganization ofaserver.OtherManagedObjectsaredetailsexpandingonthesefivebasictypes.Thesefivebasictypesare:

Folder ComputeResource ResourcePool DataCenter VirtualMachine

ItisacharacteristicofallManagedObjectsthattheyhaveaMoReftotheManagedObjectthatservesasthe parenttotheManagedObject.ThisparentMoRefallowsyoutoreconstructtheobjecthierarchyexposedby thevSphereSDK.Ingeneralthehierarchyisatreelikestructurealongthelinesof: RootFolder>DataCenter>ComputeResource>ResourcePool>VirtualMachine Therearevariationsonthistheme,dependingonwhetheryouconnecttovCenterordirectlytoanESXhost, buttheoverallorganizationislikethestructureabove.EachManagedObjectalsohasaNameproperty.

Managed Object References

AManagedObjectReference(moRef )isactuallyahandleandnottheManagedObjectitself.Whileitis absolutelycertainthataMoRefalwayscontainsauniquevalue,theuniquevalueisonlyrelativetotheinstance ofthevSpheretowhichyouareconnected.Forexample,ifavCentermanagesaclusterofESXhosts,eachESX hostmaintainsitsownManagedObjectReferencenamespaceandthevCentermustmaintainaManaged ObjectReferencenamespacerepresentingallofitsservers.So,whenanESXhostisrepresentedbyavCenter, thevCentermustensurethattheESXManagedObjectReferencesareunique.ThevCenteraccomplishesthis bycreatinguniqueManagedObjectReferencenamesinsideitsownnamespace,whichdifferfromthenames thattheESXusesforthesameManagedObjects. AvSphereinstance(vCenterorESX)attemptstokeeptheManagedObjectReferenceforavirtualmachine consistentacrosssessions,howeverconsistencyisnotalwaysguaranteed.Forexample,unregisteringand reregisteringavirtualmachinecouldresultinachangetotheManagedObjectReferenceforthevirtual machine.Hence,itisnotagoodideatostoreaMoRefvalueandexpectittoworkcorrectlyinafuturesession, oragainstadifferentvSphereinstance.

Copyright 2009 VMware, Inc. All rights reserved.

Designing Backup Solutions for VMware vSphere

Unique ID for a Virtual Machine

OnonevCenterServer,theManagedObjectReference(moRef )uniquelyidentifiesavirtualmachine.Ifyou needtoinventoryandtrackvirtualmachinebackupsacrossmultiplevCenterServers,youcanusethe instanceUuidtogetherwithmoRef.YoucanseetheinstanceUuidatthefollowingbrowserpath:
https://<vcserver>/mob/?moid=ServiceInstance&doPath=content.about

FordirectconnectionstoESX/ESXi,thehostaddressandmoRefuniquelyidentifyavirtualmachine.However thismoRefcouldbedifferentfromtheonethatvCenterServerreturns,hencethefallbacktoinstanceUuid. InstanceUuidisnewinVMwarevSphere4.0.Inpreviousreleases,thefallbackisinsteadUuid.

Gathering Status and Configuration Information

ThePropertyCollectoristhemostefficientmechanismtospecify,atthetoplevel,alloftheManagedObjects thatareofinteresttoyourapplication.Ithasmethodsforprovidingupdatesthatindicateonlychangestothe previousstateoftheseobjects.Therearetwomechanismsforacquiringtheseupdates:

PollingChecksforchanges.Theresultiseithernochangeoranobjectcontainingthechanges.The advantageofthismechanismisthatexceptforthepollrequest,itinvolvesnochatterexceptforreporting. WaitforupdatesWaitforupdatesisbasicallyablockingcalltothePropertyCollector.Thisisonly usefulifyoudedicateaprogramthreadwaitingforthecalltounblock.Thesingleadvantageofthiscall isthatthereisnochatteratallonthecommunicationsthreadunlesssomethingmustbereported.

TheVMwarevSphereWebServicesSDKProgrammingGuidecontainsalotofinformationaboutthe PropertyCollector.Thisisaverypowerfulinterface,requiringgreatattentiontodetail.Backuprelated featuresofthePropertyCollectorarecoveredinLowLevelProceduresonpage 8ofthisdocument.The nextsectionhighlightssomecharacteristicsofthePropertyCollectorasanoverview.

PropertyCollector Use
ThisdocumentassumesthatyouwanttokeepupwithchangesintheconfigurationoftheVMwareserver, andthereforeplantousetheupdatetrackingcapabilityofthePropertyCollector.ThePropertyCollector requirestwofairlycomplexarguments:thePropertySpecandtheObjectSpec. TheObjectSpeccontainsinstructionstothePropertyCollectordescribingwheretolookforthedesired data.BecauseconfigurationinformationintheVMwareserverisorganizedlikeadirectorytree,the ObjectSpecmustdescribehowtotraversethistreetoobtainthedesiredinformation.Thenetresultisa complex,nested,andrecursivelistofinstructions.Fortunately,onceyouhavedeterminedthelocationofall thedesiredinformationis,theObjectSpecneededtodeterminethelayoutofavSphereobjecthierarchycan beastaticunvaryingobject.SeethecodeexampleinsectionUnderstandinganObjectSpeconpage 9. ThePropertySpecisalistofdesiredpropertyinformation.Formulatingalistthatincludesallofthedesired informationcantakesomeefforttocompile,butoncedetermined,thiscanbeastaticobjectalso. ThedatareturnedfromthePropertyCollectorisacontainerclasscalledPropertyFilterUpdate.This classisanitembyitemlistofchangestothelistofrequestedproperties.Everyiteminthiscontaineris identifiedwithoneofthefollowingkeys:enter(add),leave(delete),andmodify.Onthefirstdatarequest, everydataitemisincluded,andeverydataitemismarkedenter. ThePropertyCollectorpresentsitsresultsinwhatamountstorandomorder.SinceallManagedObjects haveaparentproperty,youcanreconstructtheconfigurationhierarchybybuildingatreeinmemory,using theparentidentificationtoorganize.Therootfolderisidentifiedastheonlyfolderwithoutaparent.

Useful Information
YoucanfindmostoftheinformationthatisusefultoabackupprogramintheVirtualMachineManaged Object.ThisManagedObjectcontainsalotofinformation,butthefollowingisprobablymostuseful:

VirtualDisksNames,Types,andCapacities. MachineTypeandConfigurationWhateverwouldbeusefulin(re)creatingavirtualmachine.Thislist mightincludesuchinformationasnumberofCPUsandmemorysize.

Copyright 2009 VMware, Inc. All rights reserved.

Designing Backup Solutions for VMware vSphere

DisplayNamesThesenamesappearinVMwareproductssuchastheVMwarevSphereClient.You shouldkeeptrackofthesenamesandcorrelatethemforconsistencybetweenyourproductandwhat VMwaredisplaysinitsproducts.

SomethingtorememberisthatVMwaresupportsanumberofvirtualdiskimplementations.Thetypeofdisk implementationisimportantfortworeasons:

AdiskbackedbyaphysicalcompatibilityRDMmostlybypassestheESXstoragestack.Asaresult,you cannotmakeasnapshotofthistypeofvirtualdisk.Therefore,thistypeofvirtualdiskcannotbebacked upusingthesnapshotmethoddescribedinthisdocument. Onrestore,youshouldrecreatevirtualdiskwiththesamedisktypeastheoriginalvirtualmachineused.

Doing a Backup Operation

Afteryouobtaintheinformationaboutwhatisavailabletobackup,youcanperformthebackup.Thethree stepstothebackupprocessare:

CreatingaTemporarySnapshotontheTargetVirtualMachineonpage 5 ExtractingtheBackupDatafromtheTargetVirtualMachineonpage 5,saveconfigurationinformation DeletingtheTemporarySnapshotonpage 6

Creating a Temporary Snapshot on the Target Virtual Machine

ThelowlevelprocedureforcreatingasnapshotofavirtualmachineisdocumentedinthesectionCreating aSnapshotonpage 13.Theimportantthingtonoteistheflagformakingthefilesystemquiescent.Ifthefile systemisnotquiescent,youcouldendupwithasnapshotofasysteminatransitionalstate.Thediskentries maynotbeinaconsistentstate.Attemptingtorestorewiththisdatacouldprovetobedestructive. Thereisalsoasecondflagthatallowsyoutoincludeadumpofapoweredonvirtualmachinesinmemory stateinthesnapshot.Thisisnotneededforbackup,soyoushouldthisflagtofalse. Asabestpractice,youshouldsearchfor,anddelete,anyexistingsnapshotswiththesamenamethatyou selectedforthetemporarysnapshot.Thesesnapshotsarepossiblyremnantsfromfailedattemptstobackup thevirtualmachine.

Extracting the Backup Data from the Target Virtual Machine

Associatedwiththesnapshotyoujustcreatedareversionsofthevirtualdisk(s).Inordertoidentifythese disks,youobtainaMoReftothesnapshotjustcreated.FromthissnapshotMoRef,youcanextractthedisk namesandpaths.HowtodothisisdemonstratedinsectionBackingUpaVirtualDiskonpage 13. Withinaspecificsnapshot,thenamesofvirtualdiskfiles(withextension.vmdk)canbemodifiedwitha zerofilled6digitdecimalsequencenumbertoensurethatthe.vmdkfilesareuniquelynamed.Dependingon whetherornotthecurrentvirtualmachinehadapreexistingsnapshot,thedisknameforasnapshotcould havethisformat:<diskname>-<NNNNNN>.vmdk.Thisuniquenameisnolongervalidafterthesnapshotis destroyed,soanydataforasnapshotdiskshouldbestoredinthebackupprogramunderitsbasediskname. Toaccessthedatainavirtualdisk,itisnecessarytousetheVixDiskLib.Thislibraryisolatestheprogrammer fromthe(extremely)gorydetailsofextractingdatafromavirtualdiskanditsredologs.VixDiskLiballows accesstodiskdataonsectorboundariesonly,andthetransfersizeissomemultipleofthedisksectorsize. WhenaccessingdisksonESXhosts,VixDiskLibrelease1.0transferredvirtualdiskdataoverthenetwork. VixDiskLibrelease1.1containsAPIenhancementssoyoucanrequestmoreefficientdatapaths,suchasdirect SANaccessorhotaddingdiskstoavirtualbackupappliance.Theseefficientdatapathsrequireonlysmall changestoyourapplication,forexamplecallingVixDiskLib_ConnectEx()insteadofplainconnect. Partoftheinformationinavirtualdiskisanumberofkey/valuepairsthatdescribetheconfigurationofthe virtualdisk.ThisinformationcanalsobeextractedfromthevirtualdiskusingtheVixDiskLibfunction VixDiskLib_GetMetadataKeys().Youshouldsavethismetadatainformationinordertobeabletorecreate thevirtualdiskshoulditbecomenecessary.

Copyright 2009 VMware, Inc. All rights reserved.

Designing Backup Solutions for VMware vSphere

TheVixDiskLibAPIallowsabackupapplicationtoperformafullbackupofavirtualmachine.Thenewer VixMntapilibrarycanextractinformationaboutaguestoperatingsystemfromitsvirtualdisks,soyour backupapplicationcandeterminewhattypeofoperatingsystemisinstalledthere.Thisallowsmountingthe volumestodevicenodes,sothatapplicationscanperformfileorientedbackupsofavirtualmachine.

Deleting the Temporary Snapshot

Asthelastpartofthebackupprocess,youshoulddeletethetemporarysnapshot.Itisnolongerneeded,and itrepresentsstoragespacethatcouldbeputtootheruses.Theprocedureforrevertingasnapshotisoutlined intheVMwarevSphereWebServicesSDKProgrammingGuide.

Doing a Restore Operation

Therestoreprocesshastwomodesofoperation:

RestoringanExistingVirtualMachinetoaKnownStateonpage 6 CreatingaNewVirtualMachineonpage 6

Restoring an Existing Virtual Machine to a Known State

Thefollowingstepsrestoreavirtualmachinetoaknownstate: 1 Shutdownthevirtualmachine(ifitisnotalreadyshutdown). Forobvioussecurityreasons,youarenotgrantedwriteaccesstothedisksofarunningvirtualmachine. Beforeyoushutitdown,youneedtodeterminetherunstateofthevirtualmachine.Thisisinformation thatisavailablefromthePropertyCollector,andifyouhavebeenkeepingthisinformationuptodate, thenyourapplicationalreadyknowstherunstateofthevirtualmachine. TochangetherunstateofavirtualmachinerequiresthatyoudeterminetheMoRefofthevirtualmachine. YoucanthenusethisMoRefinaPowerOnVM_Taskcallthroughtheconnectionyoumaketotheserver. TheprocedureforaccomplishingvirtualmachineshutdownisoutlinedintheVMwarevSphereWeb ServicesSDKProgrammingGuideinthePerformingPowerOperationsonaVirtualMachinesection. 2 Restorethecontentsofthevirtualdisks. Theprocessofrestoringthediskdatarequiresthatyouobtainthecurrentnamesofthevirtualdisks.This processissimilartotheonedescribedinExtractingtheBackupDatafromtheTargetVirtualMachine onpage 5,exceptinthiscaseyouobtainthisinformationdirectlyfromthevirtualmachineandnotfrom asnapshot.Thetargetforthesaveddiskdatamustbetheactualdiskname(includinganysequence numberthatmayexist),becausethecurrentincarnationofthevirtualmachinemaybederivedfromone ormoresnapshots. RestoringdiskdatarequirestheuseoftheVixDiskLibinterface.Documentationofthisinterfaceisat http://www.vmware.com/support/developer/vddk/.TheVixDiskLib_Write()functionallowsyouto openthevirtualmachinesvirtualdisksandwriteyourrestoredata.VixDiskLibfunctionstransferdata toevensectorboundariesonly,andthetransferlengthmustbeanevenmultipleofasectorsize. Sincethevirtualdiskexists,itisnotnecessarytorestorethediskconfigurationinformationmentionedin ExtractingtheBackupDatafromtheTargetVirtualMachineonpage 5.

Creating a New Virtual Machine

Theprocessofbuildingavirtualmachinefrombackupdatainvolvesthefollowingsteps: 1 Createthevirtualmachine. TheprocessforcreatingavirtualmachineisdescribedintheVMwarevSphereWebServicesSDK ProgrammingGuideintheCreatingaVirtualMachinesection.Tocreateanewvirtualmachine,youmust usetheinformationaboutvirtualmachineconfigurationthatyouderivedandsavedduringthebackup process.Youshouldallowuserstheopportunitytorenamethevirtualmachineduringthisprocessincase theywanttoclonethevirtualmachine.Youalsomightconsiderofferinguserstheopportunityto changethevirtualmachineslayout(forinstance,storingthevirtualdisksondifferentdatastores).

Copyright 2009 VMware, Inc. All rights reserved.

Designing Backup Solutions for VMware vSphere

Creatingthevirtualdisksforthismachineisalsodoneatthetimeyoucreatethevirtualmachine.This processisfairlycomplicated.SeethesectionLowLevelProceduresonpage 8fordetails. 2 Restorethevirtualdiskdata. Thisprocessissimilartorestoringthecontentsofvirtualdiskswiththefollowingexception:youmust useaVixDiskLibfunctiontosetthediskconfigurationkey/valuedataintothevirtualdiskbeforewriting anybackedupdatatothedisk.ThenuseVixDiskLibtorestoredatatothedisk,asdescribedabove. 3 Poweronthevirtualmachine.

Changed Block Tracking

Thisrelativelynewfeatureprovidesthefoundationforincrementalordifferentialbackupofvirtualdisks. YourapplicationcanbackuponlychangeddataasindicatedbytheQueryChangedDiskAreasmethod. Recentlycreatedvirtualmachinescansupportthiscapability,inwhichcasethevirtualmachinecontains changeTrackingSupportedinthecapabilityfieldoftheVirtualMachineManagedObject.SeeChanged BlockTrackingonVirtualDisksonpage 21formoredetails.

Accessing Files on Virtual Disks

Itmightbenecessaryforabackupapplicationtoaccessfilesonthevirtualdisks.Youcanfindtheinterfaces toaccomplishthisintheVixMntapilibrary,associatedwithVixDiskLib.TheVixMntapiAPIallowsthedisks orvolumescontainedwithinavirtualmachinetobemountedandexaminedinanywaynecessary. Theprocedureformountingavirtualdiskrequiresthefollowingsequenceofsteps: 1 2 3 4 5 6 7 Locatethepathnamesofallthevirtualdisksassociatedwithasnapshot. CallVixDiskLib_Open()toopenallofthesevirtualdisks.ThisgivesyouanumberofVixDiskLib handles,whichyoushouldstoreinanarray. CallVixMntapi_OpenDiskSet()tocreateaVixDiskSetHandle,passinginthearrayofVixDiskLib handlesthatyoucreatedinstep2. PassVixDiskSetHandleasaparametertoVixMntapi_GetVolumeHandles()toobtainanarrayof VixVolumeHandlepointerstoallvolumesinthediskset. CallVixMntapi_GetOsInfo()todeterminewhatkindofoperatingsystemisinvolved,andwhere importantpiecesofinformationaretobefound. Forimportantvolumes,callVixMntapi_MountVolume()thenVixMntapi_GetVolumeInfo(),which revealshowthevolumeissetup. Ifyouneedinformationabouthowtheguestoperatingsystemseesthedataonthisvolume,youcanlook inthedatastructureVixVolumeInforeturnedbyVixMntapi_GetVolumeInfo(). VixVolumeInfo::symbolicLink,obtainedusingVixMntapi_GetVolumeInfo(),isthepathonthe proxywhereyoucanaccessthefilesysteminthevirtualdiskusingordinaryopen,read,andwritecalls. Onceyouaredonewithaccessingfilesinavolume,thereareVixMntapiproceduresfortakingdownthe abstractionthatyouhavecreated.Thesecallsare:

VixMntapi_DismountVolume() VixMntapi_FreeOsInfo() VixMntapi_FreeVolumeInfo() VixMntapi_CloseDiskSet()

ThisleavestheVixDiskLibhandlesthatyouobtainedinthebeginning;youmustdisposeofthoseproperly.

Copyright 2009 VMware, Inc. All rights reserved.

Designing Backup Solutions for VMware vSphere

Summary
Intheprecedingsectionsyouhaveseenatahighlevel:

Howtocontactaserver Howtoextractinformationfromthatserver Howtomanipulateitemsontheserver Howtobackupandrestoredatafromvirtualdisks Howtoaccessfiledataonvirtualdisks

Thefollowingsectionscoverthesameinformationatalowerlevel.

Low Level Procedures

Thissectiondescribesthelowleveldetailsthatshouldbehelpfulinactuallycodingyourbackupapplication. Itisnottheintentofthisdocumenttoimposeadesign.Instead,thisdocumentshouldserveasaguideline, providinghelpfulexamplesandexposition.Thecodesamplesinthissectionaredefinitelynotcomplete.They lackappropriateerrorhandling,andtheyignoreotherdetailsthatmaybenecessarytomakeaprogramwork. Thecodesamplesaremainlymeanttodemonstrateapproachestoaccomplishthebackuptask.

Communicating with the Server

Connectionstotheservermachinerequirecredentials:username,password,andhostname(orIPaddress). Thefollowingcodeshowshowtocontacttheserverandextracttheinformationthatismostusefulfor manipulatingit.ThisexampleassumesthatyouhavealreadysetupthevSphereSDKasdescribedinthe VMwarevSphereWebServicesSDKProgrammingGuidesectionsaboutinstallationandconfigurationoftheSDK server. 1 Createtheserviceinstance:
ManagedObjectReference svcRef = new ManagedObjectReference(); svcRef.setType("ServiceInstance"); svcRef.setValue("ServiceInstance");

Locatetheservice:
VimServiceLocator locator = new VimServiceLocator(); locator.setMaintainSession(true); VimPortType serviceConnection = locator.getVimPort("https://your_server/sdk"); ServiceInstanceContent serviceContent = serviceConnection.retrieveContent(svcRef); ManagedObjectReference sessionManager = serviceInstance.getSessionManager(); UserSession us = serviceConnection.login(sessionManager, username, password, null);

The PropertyCollector
ThePropertyCollectorisdescribedinsomedetailintheVMwarevSphereWebServicesSDKProgramming Guide.Thissectionappliesthosedetailstothebackuptask.

PropertyCollector Arguments
ThePropertyCollectorusestworelativelycomplicatedargumentstructures.Aswasmentionedin PropertyCollectorUseonpage 4,theseargumentsarePropertySpecandObjectSpec.ThePropertySpec isalistoftheinformationdesired,andtheObjectSpecisalistofinstructionsindicatingwherethedesired informationcanbefound.Intheory,youcanalmostdirectlyaddressanobjectusingitsMoRef.Inthatcasean ObjectSpeccouldbeverysimple.However,gettingtheMoRefinthefirstplacecanbeachallengewhena complicatedObjectSpecisrequired.ToformulatethiscomplicatedObjectSpec,youneedtounderstandthe structureoftheavailabledata,whichcanbeaconfusingexercise.Thisiscomplicatedbythefactthatan ObjectSpeccancontainrecursiveelements.

Copyright 2009 VMware, Inc. All rights reserved.

Designing Backup Solutions for VMware vSphere

Understanding an ObjectSpec AnObjectSpecisalistofObjectSpecelements.Eachelementdescribesthetypeofobject,andgivesa selectionspec.Firstconsiderthetypeofobject.MoreAboutManagedObjectsonpage 3describesfive typesofManagedObjects.Hereishowyoutraverseobjects(howoneManagedObjectleadstoanother).

FolderIfyoureadtheVMwarevSphereSDKReferenceGuidesectionaboutaFolderManagedObject,you seethatoneoftheitemscontainedintheFolderiscalledchildEntity,whichisalistofMoRefsthatcan containanyofthefollowingManagedObjecttypes:Folder,DataCenter,ComputeResource,orVirtual Machine.ThismeansthataFoldercanbeaparenttoanyoftheManagedObjectsinthislist. DataCenterDataCenterhastwoitemsthatleadtootherManagedObjects.Theseare:

hostFolderAMoReftoaFoldercontainingalistofComputeResourcescomprisingaDataCenter. vmFolderAMoReftoaFoldercontainingtheVirtualMachinesthatarepartoftheDataCenter.If itisyourobjectivetoduplicatethedisplayseeninaVirtualClientGUI,thenthisFolderisoflimited usebecauseitdoesnotdescribetheResourcePoolthatistheparentofthevirtualmachine.

ComputeResourceAComputeResourceisbasicallyhardware.AComputeResourcemaybecomposed ofmultiplehostsystems.ThishardwarerepresentsresourcesthatyoucanusetoimplementaVirtual Machineobject.However,aVirtualMachinesisalwaysachildofaResourcePool,whichisusedtocontrol thesharingoftherealmachinesresourcesamongtheVirtualMachineobjects.AComputeResource containsanitemnamedresourcePool,whichisaMoReftoaResourcePool. VirtualAppAVirtualApp(vApp)isacollectionofVirtualMachinesthatmakeupasingleapplication. ThisisaspecialformofResourcePool(definedbelow).AVirtualAppmayhavethreetypesofchildren:

VirtualMachineAfoldernamedvmcontainsalistofMoRefstochildVirtualMachines. resourcePoolAfoldercontainingalistofMoRefspointingtochildResourcePoolsorVirtualApps.

VirtualAppAVirtualAppcanbecomposedofotherVirtualApps. ResourcePoolYoucansegmenttheresourcesofaVirtualAppusingaResourcePool.

ResourcePoolResourcePoolcontainstwochilditems:

resourcePoolAfoldercontainingalistofMoRefspointingtochildResourcePoolsorVirtualApps. vmAlistofMoRefstochildVirtualMachinesthatemploytheresourcesoftheparentResourcePool. AVirtualMachinealwayslistsaResourcePoolasitsparent.

VirtualMachineVirtualMachinecanbeconsideredtobeanendobjectandassuchyouneednot describeanytraversalforthisobject.

YoumustunderstandthattheObjectSpecdoesnothavetoleadyouanyfartherthantheMoRefofatarget object.YoucangatherinformationabouttheManagedObjectitselfusingtheMoRefandthePropertySpec. ThisisdescribedindetailinthesectionUnderstandingaPropertySpeconpage 10. InthedocumentVMwarevSphereWebServicesSDKProgrammingGuide,anObjectSpecelement(calleda TraversalSpec)isdescribedascontainingthefollowingelements:

TypeThetypeofobjectbeingreferenced. PathTheelementcontainedintheobjectthatisusedtosteertraversal. NameOptionalreferencenamethatyoucanusetoreferencethisTraversalSpecinanotherSelectSet. SelectSetAnarraycontainingeitherSelectionSpecorTraversalSpecelements.

SelectionSpecisadirecttargetfortraversal,asisTraversalSpec(aclassextendingSelectionSpec).Itis intheSelectSetthatrecursioncanoccur. Ifyouwishtotraversetheentireconfigurationtreeforaserver,thenyouneedonlytherootnodeMoRef, whichisalwaysaFolder.ThisrootfoldernameisavailablebyusingthefunctiongetRootFolderfromthe serviceinstancethatconnectstothevSphere.AlloftheabovegoesintothisJavacodesample:

// Remember, TraversalSpec objects can use a symbolic name.

Copyright 2009 VMware, Inc. All rights reserved.

Designing Backup Solutions for VMware vSphere

// In this case we use the symbolic name "folderTSpec". // First we must define the TraversalSpec objects used to fill in the ObjectSpec. // // This TraversalSpec traverses Datacenter to vmFolder TraversalSpec dc2vmFolder = new TraversalSpec(); dc2vmFolder.setType("Datacenter"); // Type of object for this spec dc2vmFolder.setPath("vmFolder"); // Property name defining the next object dc2vmFolder.setSelectSet(new SelectionSpec[] {"folderTSpec"}); // // This TraversalSpec traverses Datacenter to hostFolder TraversalSpec dc2hostFolder = new TraversalSpec(); dc2hostFolder.setType("Datacenter"); dc2hostFolder.setPath("hostFolder"); dc2vmFolder.setSelectSet(new SelectionSpec[] {"folderTSpec"}); // // This TraversalSpec traverses ComputeResource to resourcePool TraversalSpec cr2resourcePool = new TraversalSpec(); cr2resourcePool.setType("ComputeResource"); cr2resourcePool.setPath("resourcePool"); // // This TraversalSpec traverses ComputeResource to host TraversalSpec cr2host = new TraversalSpec(); cr2host.setType("ComputeResource"); cr2host.setPath("host"); // // This TraversalSpec traverses ResourcePool to resourcePool TraversalSpec rp2rp = new TraversalSpec(); rp2rp.setType("ResourcePool"); rp2rp.setPath("resourcePool"); // // Finally, we tie it all together with the Folder TraversalSpec TraversalSpec folderTS = new TraversalSpec(); folderTS.setName{"folderTSpec"); // Used for symbolic reference folderTS.setType("Folder"); folderTS.setPath("childEntity"); folderTS.setSelectSet(new SelectionSpec[]{ "folderTSpec", dc2vmFolder, dc2hostFolder, cr2resourcePool, rp2rp}); ObjectSpec ospec = new ObjectSpec(); ospec.setObj(startingPoint); // This is where you supply the starting MoRef (usually root folder) ospec.setSkip(Boolean.FALSE); ospec.setSelectSet(folderTS); // Attach the TraversalSpec we designed above

Understanding a PropertySpec APropertySpecisalistofindividualpropertiesthatcanbefoundatplacesidentifiedbytheObjectSpec. OncethePropertyCollectorhasaMoRef,itcanthenreturnthepropertiesassociatedwiththatMoRef. Thiscanincludenestedproperties.Nestedpropertiesarepropertiesthatcanbefoundinsideofproperties identifiedatthetopleveloftheManagedObject.Nestedpropertiesareidentifiedbyadotnotation. AnexampleofnestedpropertiescanbedrawnfromtheVirtualMachineManagedObject.Ifyouexaminethe VMwarevSphereSDKReferenceGuideyouseethataVirtualMachinehasthepropertyidentifiedassummary, whichidentifiesaVirtualMachineSummaryManagedObject.Inturn,theVirtualMachineSummarycontains propertyconfig,whichidentifiesaVirtualMachineConfigSummary.VirtualMachineConfigSummaryhas apropertycalledname,whichisastringcontainingthedisplaynameoftheVirtualMachine.Youcanaccess thisfinalpropertyusingstringvaluesummary.config.name.ForalltheVirtualMachineConfigSummary information,stringvaluesummary.configcausesreturnofallVirtualMachineConfigSummaryproperties. ThePropertyCollectorrequiresanarrayofPropertySpecelements.Eachelementincludes:

TypeThetypeofobjectthatidentifiestheenclosedlistofproperties. PathSetAnarrayofstringscontainingnamesofpropertiestobereturned,includingnestedproperties.

Copyright 2009 VMware, Inc. All rights reserved.

10

Designing Backup Solutions for VMware vSphere

Itisnecessarytoaddanelementforeachtypeofobjectthatyouwishtoqueryforproperties.Thefollowingis acodesampleofaPropertySpec:
// This code demonstrates how to specify a PropertySpec for several types of target objects: PropertySpec folderSp = new PropertySpec(); folderSp.setType("Folder"); folderSp.setAll(Boolean.FALSE); folderSp.setPathSet(new String [] {"parent", "name"}); PropertySpec dcSp = new PropertySpec(); dcSp.setType("Datacenter"); dcSp.setAll(Boolean.FALSE); dcSp.setPathSet(new String [] {"parent","name"}); PropertySpec rpSp = new PropertySpec(); rpSp.setType("ResourcePool"); rpSp.setAll(Boolean.FALSE); rpSp.setPathSet(new String [] {"parent","name","vm"}); PropertySpec crSp = new PropertySpec(); crSp.setType("ComputeResource"); crSp.setAll(Boolean.FALSE); crSp.set:PathSet(new String [] {"parent","name"}); PropertySpec vmSp = new PropertySpec(); vmSp.setType("VirtualMachine"); vmSp.setAll(Boolean.FALSE); vmSp.setPathSet(new String [] {"parent", "name", "summary.config", "snapshot", "config.hardware.device"}); // Tie it all together PropertySpec [] pspec = new PropertySpec [] {folderSp, dcSp, rpSp, crSp, vmSp};

Getting the Data from the PropertyCollector

NowthatwehavedefinedObjectSpecandPropertySpec(thewhereandwhat),weneedtoputtheminto aFilterSpecthatcombinesthetwo.AnarrayofFilterSpecelementsispassedtothePropertyCollector (theminimumnumberofelementsisone).TwomechanismscanretrievedatafromPropertyCollector:

RetrievePropertiesAonetimerequestforallofthedesiredproperties. UpdatesPropertyCollectorupdaterequeststaketwoforms:pollingandwaiting,discussedbelow.

Checking for Updates TheRetrievePropertiesoperationisratherobvious,soconsidertheupdatemethod.IneitherPollingor Waiting,itisfirstnecessarytoregisteryourFilterSpecarrayobjectwiththePropertyCollector.You accomplishthisusingtheCreateFilterfunction,whichsendsacopyofyourFilterSpectotheserver. UnliketheRetrievePropertiesfunction,FilterSpecisnotdiscardedaftertheCreateFilteroperation. ThefollowingcodeshowshowtosetyourFilterSpec:

// We already showed examples of creating pspec and ospec in the examples above. // Remember, the PropertyCollector wants an array of FilterSpec objects, so: PropertyFilterSpec fs = new PropertyFilterSpec(); fs.setPropSet(pspec); fs.setObjectSet(ospec); PropertyFilterSpec [] fsa = new PropertyFilterSpec [] {fs}; ManagedObjectReference pcRef = serviceContent.getPropertyCollector(); // This next statement sends the filter to the server for reference by the PropertyCollector ManagedObjectReference pFilter = serviceConnection.CreateFilter(pcRef, fsa, Boolean.FALSE);

Copyright 2009 VMware, Inc. All rights reserved.

11

Designing Backup Solutions for VMware vSphere

Ifyouwishtobeginpolling,youmaythencallthefunctionCheckForUpdates,whichonthefirsttry(whenit mustcontainanemptystringfortheversionnumber)returnsacompletedumpofalltherequestedproperties, alongwithaversionnumber.SubsequentcallstoCheckForUpdatesmustcontainthisversionnumberto indicatetothePropertyCollectorthatyouseekanychangesthatdeviatefromthisversion.Theresultiseither apartiallistcontainingonlythechangesfromthepreviousversion(includinganewversionnumber),ora returncodeindicatingnodatahaschanged.Thefollowingcodesampleshowshowtocheckforupdates:

String updateVersion = ""; // Start with no version UpdateSet changeData = serviceConnection.CheckForUpdates(pcRef, version); if (changeData != nil) { version = changeData.getVersion(); // Extract the version of the data set } ... // Get changes since the last version was sent. UpdateSet latestData = serviceConnection.CheckForUpdates(pcRef, version);

Ifinsteadyouwishtowaitforupdatestooccur,youmustcreateataskthreadthatblocksonthecall WaitForUpdates.Thistaskreturnschangesonlyastheyoccurandnotatanyothertime. Extracting Information from the Change Data ThedatareturnedfromCheckForUpdates(orWaitForUpdates)isanarrayofPropertyFilterUpdate entries.SinceaPropertyFilterUpdateentryisverygeneric,hereissomecodeshowinghowtoextract informationfromthePropertyFilterUpdate.
// Extract the PropertyFilterUpdate set from the changeData // PropertyFilterUpdate [] updateSet = changeData.getFilterSet(); // // There is one entry in the updateSet for each filter you registered with the PropertyCollector. // Since we currently have only one filter, the array length should be one. // PropertyFilterUpdate myUpdate = updateSet[0]; ObjectUpdate [] changes = myUpdate.getObjectSet(); for (a = 0; a < changes.length; ++a) { ObjectUpdate theObject = changes[a]; String objName = theObject.getObj().getMoType().getName(); // Must decide how to handle the value based on the name returned. // The only names returned are names found in the PropertySpec lists. ... }

Getting Specific Data Fromtimetotime,youmightneedtogetdatathatisrelevanttoasingleitem.Inthatcaseyoucancreatea simpleObjectSpecnamingtheMoReffortheitemofinterest.ThePropertySpeccanthenbesettoobtainthe propertiesyouwant,andyoucanuseRetrievePropertiestogetthedata.YoucanobtainMoReffromyour generalexaminationofthepropertiesoutlinedabove(whereyousearchforinformationfromtherootFolder).

Identifying Virtual Disks for Backup and Restore

Whenattemptingtobackupavirtualmachine,youfirstneedtocreateasnapshot.Oncethesnapshotis created,youthenneedtofindandidentifythevirtualdisksassociatedwiththissnapshot.Ifyouare attemptingtorestorethevirtualdisksinthevirtualmachine,thenyouneedtofindandidentifythevirtual disksthatarecurrentlyactive.Tounderstandwhythismightbeaproblem,itisfirstnecessarytounderstand thatavirtualmachinemayhavemultiplesnapshotsassociatedwithit.Eachsnapshothasavirtualcopyof thevirtualdisksforthevirtualmachine.Thesecopiesarenamedwiththebasenameofthedisk,anda uniquedecimalnumberappendedtothename.Theformatofthenumberisahyphencharacterfollowedby a6digitzerofillednumber.Anexamplediskcopynamewouldbe:mydisk-NNNNNN.vmdkwhereNNNNNN wouldbesomenumberlike:000032.

Copyright 2009 VMware, Inc. All rights reserved.

12

Designing Backup Solutions for VMware vSphere

Inadditiontothenameofthedisk,thereshouldbeapathtothisdiskontheserver,whichisstoredasa datastorepath:[storagex]myvmname/mydisk-NNNNNN.vmdk.Thepathcomponentinthesquarebrackets correspondstothenameofthedatastorethatcontainsthisvirtualdiskwhiletheremainderofthepathstring representsthelocationrelativetotherootofthisdatastore.Usuallyyoudonothavetobeconcernedwiththis, sincethevSphereSDKtakescareofthis.Sohowdoyouobtainthispath?Whetherwearetalkingabouta snapshotorobtainingtheinformationfromthebasevirtualmachine,theprocessisthesame.Theonly differenceisintheManagedObjectthatweusetoextracttheinformation. WhetheryouusethePropertyCollectortogetaVirtualMachineoraVirtualMachineSnapshot,youneed toselecttheproperty:config.hardware.device.ThisreturnsanarrayofVirtualDevicesassociatedwiththe VirtualMachineorSnapshot.YoumustscanthislistofdevicestoextracttheVirtualDisks.Allthatisnecessary istoseeifeachVirtualDeviceentryextendstoVirtualDisk.Whenyoufindsuchanentry,examinethe BackingInfoproperty.Youmustextendthetypeofthebackingpropertytooneofthefollowing:

VirtualDiskFlatVer1BackingInfo VirtualDiskFlatVer2BackingInfo VirtualDiskRawDiskMappingVer1BackingInfo VirtualDiskSparseVer1BackingInfo VirtualDiskSparseVer2BackingInfo

ItisimportanttoknowwhichbackingtypeisinuseinordertobeabletorecreatetheVirtualDisk.Itisalso importanttoknowthatyoucannotsnapshotadiskoftypeVirtualDiskRawDiskMappingVer1BackingInfo, andthereforeyoucannotbackupthistypeofVirtualDisk. ThepropertiesofinterestarethebackingfileNameandtheVirtualDiskcapacityInKB.Additionally,when changetrackingisinplace,youshouldalsosavethechangeID.

Creating a Snapshot
Inordertoperformabackupoperation,youmustfirstcreateasnapshotofthetargetvirtualmachine.The followingcodesampleshowshowtocreateasnapshotonaspecificvirtualmachine:
// At this point we assume the virtual machine is identified as ManagedObjectReference vmMoRef. String SnapshotName = "Backup"; String SnapshotDescription = "Temporary Snapshot for Backup"; boolean memory_files = false; boolean quiesce_filesystem = true; ManagedObjectReference taskRef = serviceConnection.CreateSnapshot_Task(vmMoRef, SnapshotName, SnapshotDescription, memory_files, quiesce_filesystem);

Backing Up a Virtual Disk

ThissectiondescribeshowtogetdatafromtheVirtualDiskafteryouhaveidentifiedit.Inordertoaccessa virtualdisk,youmustusetheVixDiskLib.ThefollowingcodeshowshowtoinitializetheVixDiskLiband useitforaccessingavirtualdisk.AlloperationsrequireaVixDiskLibconnectiontoaccessvirtualdiskdata. AtthepresenttimeVixDiskLibisimplementedonlyfortheClanguage,sothisisnotJavacode:
VixDiskLibConnectParams connectParams; VixDiskLibConnection srcConnection; connectParams.serverName = strdup("TargetServer"); connectParams.creds.uid.userName = strdup("root"); connectParams.creds.uid.password = strdup("yourPasswd"); connectParams.port = 902; VixError vixError = VixDiskLib_Init(1, 0, &logFunc, &warnFunc, &panicFunc, libDir); vixError = VixDiskLib_Connect(&connectParams, &srcConnection);

Thisnextsectionofcodeshowshowtoopenandreadaspecificvirtualdisk:
VixDiskLibHandle diskHandle; vixError = VixDiskLib_Open(srcConnection, diskPath, flags, &diskHandle); uint8 mybuffer[some_multiple_of_512]; vixError = VixDiskLib_Read(diskHandle, startSector, numSectors, &mybuffer);

Copyright 2009 VMware, Inc. All rights reserved.

13

Designing Backup Solutions for VMware vSphere

// Also getting the disk metadata: size_t requiredLength = 1; char *buf = new char [1]; // This next operation fails, but updates "requiredLength" with the proper buffer size vixError = VixDiskLib_GetMetadataKeys(diskHandle, buf, requiredLength, &requiredLength); delete [] buf; buf = new char[requiredLength]; // Create a large enough buffer vixError = VixDiskLib_GetMetadataKeys(diskHandle, buf, requiredLength, NULL);

Andfinally,closethediskHandle:
vixError = VixDiskLib_Close(diskHandle); // And if you are completely done with the VixDiskLib VixDiskLib_Disconnect(srcConnection); VixDiskLib_Exit();

Deleting a Snapshot
Whenyouaredoneperformingabackup,youneedtodeletethetemporarysnapshot.Youshouldalready havetheMoRefforthesnapshotfromtheoperationthatcreatedthesnapshotinthefirstplace.Thefollowing Javacodedemonstrateshowtodeletethesnapshot:
ManagedObjectReference removeSnapshotTask; SnapshotManagedObject snapshot; removeSnapshotTask = serviceConnection.removeSnapshot_Task(snapshot, Boolean FALSE);

Restoring a Virtual Disk

ThissectiondescribeshowtorestorethedataintotheVirtualDisk. Nomatterhowyouattempttodoit,youcannotgetwriteaccesstoaVirtualDiskthatisinactiveuse.Youfirst mustensurethatthediskisnotinusebyhaltingtheparentVirtualMachine,thenperformingthepoweroff sequence.ThefollowingcodesampledemonstrateshowtopoweroffaVirtualMachine:
// At this point we assume that you have a ManagedObjectReference to the VM - vmMoRef. // You also need a ManagedObjectReference to the host running the VM - hostMoRef. ManagedObjectReference taskRef = serviceConnection.powerOffVm(vmMoRef, hostMoRef);

ThisisthephasewhereitisnecessarytouseVixDiskLibtoreloadthecontentsoftheVirtualDisk,sothe followingcodeisC,notJava:
// At this point we assume that you already have a VixDiskLib connection to the server machine. uint8 mybuffer[some_multiple_of_512]; int mylocalfile = open("localfile", openflags); read(mylocalfile, mybuffer, sizeof mybuffer); vixError = VixDiskLib_Open(srcConnection, path, flags, &diskHandle); VixDiskLib_Write(diskHandle, startsector, (sizeof mybuffer) / 512, mybuffer);

Creating a Virtual Machine

ThissectionshowshowtocreateaVirtualMachineobject.TheprocessofcreatingaVirtualMachineisfairly complicated.InordertoaccomplishthisitisnecessarytocreateaVirtualMachineConfigSpecdescribing theVirtualMachineandallofitssupportingvirtualdevices.Almostalloftherequiredinformationisavailable fromtheVirtualMachinepropertyconfig.hardware.device,whichisatablecontainingallofthedevice configurationinformation.Therelationshipsbetweenthedevicesaredescribedbythevaluekey,whichisa uniqueidentifierforthedevice.Inturn,eachdevicehasacontrollerKey,whichisthekeyidentifierofthe controllerwherethedeviceisconnected.Itisanecessarypracticetospecifythecontroller/keyrelationships usingnegativekeynumbers.Thisguaranteesthatthekeynumberdoesnotconflictwiththerealkeynumber whenitisassignedbytheserver.BelowisanactualVirtualMachineConfigSpecthatwasusedtocreatea VirtualMachine(vim.vm.ConfigSpec).
// beginning of VirtualMachineConfigSpec // ends several pages later { dynamicType = <unset>, changeVersion = <unset>,

Copyright 2009 VMware, Inc. All rights reserved.

14

Designing Backup Solutions for VMware vSphere

//This is the display name of the VM name = My New VM, version = "vmx-04", uuid = <unset>, instanceUuid = <unset>, npivWorldWideNameType = <unset>, npivDesiredNodeWwns = <unset>, npivDesiredPortWwns = <unset>, npivTemporaryDisabled = <unset>, npivOnNonRdmDisks = <unset>, npivWorldWideNameOp = <unset>, locationId = <unset>, // // This is advisory, the disk determines the O/S guestId = "winNetStandardGuest", alternateGuestName = "Microsoft Windows Server 2003, Standard annotation = <unset>, files = (vim.vm.FileInfo) { dynamicType = <unset>, vmPathName = "[plat004-local]", snapshotDirectory = "[plat004-local]", suspendDirectory = <unset>, logDirectory = <unset>, }, tools = (vim.vm.ToolsConfigInfo) { dynamicType = <unset>, toolsVersion = <unset>, afterPowerOn = true, afterResume = true, beforeGuestStandby = true, beforeGuestShutdown = true, beforeGuestReboot = true, toolsUpgradePolicy = <unset>, pendingCustomization = <unset>, syncTimeWithHost = <unset>, }, flags = (vim.vm.FlagInfo) { dynamicType = <unset>, disableAcceleration = <unset>, enableLogging = <unset>, useToe = <unset>, runWithDebugInfo = <unset>, monitorType = <unset>, htSharing = <unset>, snapshotDisabled = <unset>, snapshotLocked = <unset>, diskUuidEnabled = <unset>, virtualMmuUsage = <unset>, snapshotPowerOffBehavior = "powerOff", recordReplayEnabled = <unset>, }, consolePreferences = (vim.vm.ConsolePreferences) null, powerOpInfo = (vim.vm.DefaultPowerOpInfo) { dynamicType = <unset>, powerOffType = "preset", suspendType = "preset", resetType = "preset", defaultPowerOffType = <unset>, defaultSuspendType = <unset>, defaultResetType = <unset>, standbyAction = "powerOnSuspend", }, // the number of CPUs numCPUs = 1, // the number of memory megabytes memoryMB = 256, memoryHotAddEnabled = <unset>, cpuHotAddEnabled = <unset>,

Edition (32-bit)",

Copyright 2009 VMware, Inc. All rights reserved.

15

Designing Backup Solutions for VMware vSphere

//

//

//

//

//

//

//

cpuHotRemoveEnabled = <unset>, deviceChange = (vim.vm.device.VirtualDeviceSpec) [ (vim.vm.device.VirtualDeviceSpec) { dynamicType = <unset>, operation = "add", fileOperation = <unset>, CDROM device = (vim.vm.device.VirtualCdrom) { dynamicType = <unset>, key number of CDROM key = -42, deviceInfo = (vim.Description) null, backing = (vim.vm.device.VirtualCdrom.RemotePassthroughBackingInfo) { dynamicType = <unset>, deviceName = "", useAutoDetect = <unset>, exclusive = false, }, connectable = (vim.vm.device.VirtualDevice.ConnectInfo) { dynamicType = <unset>, startConnected = false, allowGuestControl = true, connected = false, }, connects to this controller controllerKey = 200, unitNumber = 0, }, }, (vim.vm.device.VirtualDeviceSpec) { dynamicType = <unset>, operation = "add", fileOperation = <unset>, SCSI controller device = (vim.vm.device.VirtualLsiLogicController) { dynamicType = <unset>, key number of SCSI controller key = -44, deviceInfo = (vim.Description) null, backing = (vim.vm.device.VirtualDevice.BackingInfo) null, connectable = (vim.vm.device.VirtualDevice.ConnectInfo) null, controllerKey = <unset>, unitNumber = <unset>, busNumber = 0, hotAddRemove = <unset>, sharedBus = "noSharing", scsiCtlrUnitNumber = <unset>, }, }, (vim.vm.device.VirtualDeviceSpec) { dynamicType = <unset>, operation = "add", fileOperation = <unset>, Network controller device = (vim.vm.device.VirtualPCNet32) { dynamicType = <unset>, key number of Network controller key = -48, deviceInfo = (vim.Description) null, backing = (vim.vm.device.VirtualEthernetCard.NetworkBackingInfo) { dynamicType = <unset>, deviceName = "Virtual Machine Network", useAutoDetect = <unset>, network = <unset>, inPassthroughMode = <unset>, }, connectable = (vim.vm.device.VirtualDevice.ConnectInfo) { dynamicType = <unset>,

Copyright 2009 VMware, Inc. All rights reserved.

16

Designing Backup Solutions for VMware vSphere

startConnected = true, allowGuestControl = true, connected = true, }, controllerKey = <unset>, unitNumber = <unset>, addressType = "generated", macAddress = <unset>, wakeOnLanEnabled = true, }, }, (vim.vm.device.VirtualDeviceSpec) { dynamicType = <unset>, operation = "add", fileOperation = "create", SCSI disk one device = (vim.vm.device.VirtualDisk) { dynamicType = <unset>, key number for SCSI disk one key = -1000000, deviceInfo = (vim.Description) null, backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) { dynamicType = <unset>, fileName = "", datastore = <unset>, diskMode = "persistent", split = false, writeThrough = false, thinProvisioned = <unset>, eagerlyScrub = <unset>, uuid = <unset>, contentId = <unset>, changeId = <unset>, parent = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) null, }, connectable = (vim.vm.device.VirtualDevice.ConnectInfo) { dynamicType = <unset>, startConnected = true, allowGuestControl = false, connected = true, }, controller for SCSI disk one controllerKey = -44, unitNumber = 0, size in MB SCSI disk one capacityInKB = 524288, committedSpace = <unset>, shares = (vim.SharesInfo) null, }, }, (vim.vm.device.VirtualDeviceSpec) { dynamicType = <unset>, operation = "add", fileOperation = "create", SCSI disk two device = (vim.vm.device.VirtualDisk) { dynamicType = <unset>, key number of SCSI disk two key = -100, deviceInfo = (vim.Description) null, backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) { dynamicType = <unset>, fileName = "", datastore = <unset>, diskMode = "persistent", split = false, writeThrough = false, thinProvisioned = <unset>,

//

//

//

//

//

//

Copyright 2009 VMware, Inc. All rights reserved.

17

Designing Backup Solutions for VMware vSphere

eagerlyScrub = <unset>, uuid = <unset>, contentId = <unset>, changeId = <unset>, parent = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) null, }, connectable = (vim.vm.device.VirtualDevice.ConnectInfo) { dynamicType = <unset>, startConnected = true, allowGuestControl = false, connected = true, }, // controller for SCSI disk two controllerKey = -44, unitNumber = 1, // size in MB SCSI disk two capacityInKB = 131072, committedSpace = <unset>, shares = (vim.SharesInfo) null, }, } }, cpuAllocation = (vim.ResourceAllocationInfo) { dynamicType = <unset>, reservation = 0, expandableReservation = <unset>, limit = <unset>, shares = (vim.SharesInfo) { dynamicType = <unset>, shares = 100, level = "normal", }, overheadLimit = <unset>, }, memoryAllocation = (vim.ResourceAllocationInfo) { dynamicType = <unset>, reservation = 0, expandableReservation = <unset>, limit = <unset>, shares = (vim.SharesInfo) { dynamicType = <unset>, shares = 100, level = "normal", }, overheadLimit = <unset>, }, cpuAffinity = (vim.vm.AffinityInfo) null, memoryAffinity = (vim.vm.AffinityInfo) null, networkShaper = (vim.vm.NetworkShaperInfo) null, swapPlacement = <unset>, swapDirectory = <unset>, preserveSwapOnPowerOff = <unset>, bootOptions = (vim.vm.BootOptions) null, appliance = (vim.vService.ConfigSpec) null, ftInfo = (vim.vm.FaultToleranceConfigInfo) null, applianceConfigRemoved = <unset>, vAssertsEnabled = <unset>, changeTrackingEnabled = <unset>, } // end of VirtualMachineConfigSpec

Theinformationaboveappearstobequitecomplex,butmuchoftheinputconsistsofdefaultedvaluestobe assignedbythesystem.Theremainderofthesuppliedinformationcanbeextractedfromtheoutputofthe config.hardware.devicetablefromthePropertyCollector.Borrowingheavilyfromthecodeexamplein theVMwarevSphereWebServicesSDKProgrammingGuide,wegetthefollowingcodethatduplicates functionalityoftheconfigurationspecificationseenabove:

// Duplicate virtual machine configuration

Copyright 2009 VMware, Inc. All rights reserved.

18

Designing Backup Solutions for VMware vSphere

VirtualMachineConfigSpec configSpec = new VirtualMachineConfigSpec(); // Set the VM values configSpec.setName("My New VM"); configSpec.setVersion("vmx-04"); configSpec.setGuestId("winNetStandardGuest"); configSpec.setNumCPUs(1); configSpec.setMemoryMB(256); // Set up file storage info VirtualMachineFileInfo vmfi = new VirtualMachineFileInfo(); vmfi.setVmPathName("[plat004-local]"); configSpec.setFiles(vmfi); vmfi.setSnapshotDirectory("[plat004-local]"); // Set up tools config info ToolsConfigInfo tools = new ToolsConfigInfo(); configSpec.setTools(tools); tools.setAfterPowerOn(new Boolean(true)); tools.setAfterResume(new Boolean(true)); tools.setBeforeGuestStandby(new Boolean(true)); tools.setBeforeGuestShutdown(new Boolean(true)); tools.setBeforeGuestReboot(new Boolean(true)); // Set flags VirtualMachineFlagInfo flags = new VirtualMachineFlagInfo(); configSpec.setFlags(flags); flags.setSnapshotPowerOffBehavior("powerOff"); // Set power op info VirtualMachineDefaultPowerOpInfo powerInfo = new VirtualMachineDefaultPowerOpInfo(); configSpec.setPowerOpInfo(powerInfo); powerInfo.setPowerOffType("preset"); powerInfo.setSuspendType("preset"); powerInfo.setResetType("preset"); powerInfo.setStandbyAction("powerOnSuspend"); // Now add in the devices VirtualDeviceConfigSpec[] deviceConfigSpec = new VirtualDeviceConfigSpec [5]; configSpec.setDeviceChange(deviceConfigSpec); // Formulate the CDROM deviceConfigSpec[0].setOperation(VirtualDeviceConfigSpecOperation.add); VirtualCdrom cdrom = new VirtualCdrom(); VirtualCdromIsoBackingInfo cdDeviceBacking = new VirtualCdromRemotePassthroughBackingInfo(); cdDeviceBacking.setDatastore(datastoreRef); cdrom.setBacking(cdDeviceBacking); cdrom.setKey(-42); cdrom.setControllerKey(new Integer(200)); cdrom.setUnitNumber(new Integer(0)); deviceConfigSpec[0].setDevice(cdrom); // Formulate the SCSI controller deviceConfigSpec[1].setOperation(VirtualDeviceConfigSpecOperation.add); VirtualLsiLogicController scsiCtrl = new VirtualLsiLogicController(); scsiCtrl.setBusNumber(0); deviceConfigSpec[1].setDevice(scsiCtrl); scsiCtrl.setKey(-44); scsiCtrl.setSharedBus(VirtualSCSISharing.noSharing); // Formulate SCIS disk one deviceConfigSpec[2].setFileOperation(VirtualDeviceConfigSpecFileOperation.create); deviceConfigSpec[2].setOperation(VirtualDeviceConfigSpecOperation.add); VirtualDisk disk = new VirtualDisk(); VirtualDiskFlatVer2BackingInfo diskfileBacking = new VirtualDiskFlatVer2BackingInfo(); diskfileBacking.setDatastore(datastoreRef); diskfileBacking.setFileName(volumeName); diskfileBacking.setDiskMode("persistent"); diskfileBacking.setSplit(new Boolean(false)); diskfileBacking.setWriteThrough(new Boolean(false)); disk.setKey(-1000000); disk.setControllerKey(new Integer(-44)); disk.setUnitNumber(new Integer(0)); disk.setBacking(diskfileBacking); disk.setCapacityInKB(524288); deviceConfigSpec[2].setDevice(disk); // Formulate SCSI disk two

Copyright 2009 VMware, Inc. All rights reserved.

19

Designing Backup Solutions for VMware vSphere

deviceConfigSpec[3].setFileOperation(VirtualDeviceConfigSpecFileOperation.create); deviceConfigSpec[3].setOperation(VirtualDeviceConfigSpecOperation.add); VirtualDisk disk2 = new VirtualDisk(); VirtualDiskFlatVer2BackingInfo diskfileBacking2 = new VirtualDiskFlatVer2BackingInfo(); diskfileBacking2.setDatastore(datastoreRef); diskfileBacking2.setFileName(volumeName); diskfileBacking2.setDiskMode("persistent"); diskfileBacking2.setSplit(new Boolean(false)); diskfileBacking2.setWriteThrough(new Boolean(false)); disk2.setKey(-100); disk2.setControllerKey(new Integer(-44)); disk2.setUnitNumber(new Integer(1)); disk2.setBacking(diskfileBacking2); disk2.setCapacityInKB(131072); deviceConfigSpec[3].setDevice(disk2); // Finally, formulate the NIC deviceConfigSpec[4].setOperation(VirtualDeviceConfigSpecOperation.add); com.VMware.vim.VirtualEthernetCard nic = new VirtualPCNet32(); VirtualEthernetCardNetworkBackingInfo nicBacking = new VirtualEthernetCardNetworkBackingInfo(); nicBacking.setNetwork(networkRef); nicBacking.setDeviceName(networkName); nic.setAddressType("generated"); nic.setBacking(nicBacking); nic.setKey(-48); deviceConfigSpec[4].setDevice(nic); // Now that it is all put together, create the virtual machine // Note that folderMo, hostMo, and resourcePool are MoRefs to the Folder, Host, and ResourcePool // where the VM is to be created ManagedObjectReference taskMoRef = serviceConnection.getService().createVM_Task( folderMo, configSpec, resourcePool, hostMo);

Using VirtualMachineConfigInfo to Create a Virtual Machine

AbackupapplicationcanalsousetheinformationcontainedinaVirtualMachineConfigInfo.Ifyou preservealloftheinformationinaVirtualMachineConfigInfothatdescribesthevirtualmachine,youcan transfermuchofthisinformationintoaVirtualMachineConfigSpecinordertocreateavirtualmachine. However,someoftheinformationinVirtualMachineConfigInfoisnotneeded,andifusedintheSpec, virtualmachinecreationcanfail.Forexample,aVirtualMachineConfigSpecthatcontainsinformation aboutsocalledDefaultDevicesusuallyfails.Thelistofdefaultdevicescontains:
vim.vm.device.VirtualIDEController vim.vm.device.VirtualPS2Controller vim.vm.device.VirtualPCIController vim.vm.device.VirtualSIOController vim.vm.device.VirtualKeyboard vim.vm.device.VirtualVMCIDevice vim.vm.device.VirtualPointingDevice

However,othercontrollersanddevicesmustbeexplicitlyincludedintheVirtualMachineConfigSpec. Someinformationaboutdevicesisalsounneededandcancauseproblemsifsupplied.Eachcontrollerdevice hasitsvim.vm.device.VirtualController.devicefield,whichisanarrayofdevicesthatreporttothe controller.Theserverrebuildsthislistwhenavirtualmachineiscreatedusingthe(negative)devicekey numberssuppliedasaguide. Therelationshipbetweencontrolleranddevicemustbepreservedusingnegativekeynumbersinthesame relationshipasinthehardwarearrayoftheVirtualMachineConfigInfo. OneotherfeatureofaVirtualMachineConfigInforequiressubstitution.VirtualMachineConfigInfo containsthefieldcpuFeatureMask,whichisanarrayofHostCpuIdInfo.Theentriesinthisarraymustbe convertedtoArrayUpdateSpecentriesthatcontaintheVirtualMachineCpuIdInfoSpecalongwithafield calledoperationthatmustcontainthevalueArrayUpdateOperation::add. TheVirtualMachineCpuIdInfoSpecalsocontainsaHostCpuIdInfothatyoucancopyfromthe cpuFeatureMaskarrayinVirtualMachineConfigInfo. EverythingelsecanbecopiedintactfromtheVirtualMachineConfigInfodata.

Copyright 2009 VMware, Inc. All rights reserved.

20

Designing Backup Solutions for VMware vSphere

Restoring the Data

AsinthesectionRestoringaVirtualDiskonpage 14,VixDiskLibfunctionsprovideinterfacesforwriting thedatatovirtualdisk,eitherlocallyorremotely.

Changed Block Tracking on Virtual Disks

OnhostsrunningESX4.0andlater,virtualmachinescankeeptrackofdisksectorsthathavechanged.Thisis calledChangedBlockTracking.ItsmethodintheVMwarevSphereAPIisQueryChangedDiskAreas,which takesthefollowingparameters:

_thisManagedObjectReferencetothevirtualmachine. snapshotManagedObjectReferencetoaSnapshotofthevirtualmachine. deviceKeyVirtualdiskforwhichtocomputethechanges. startOffsetByteoffsetwheretostartcomputingchangestovirtualdisk.Thelengthofvirtualdisk sector(s)examinedisreturnedinDiskChangeInfo. changeIdAnidentifierforthestateofavirtualdiskataspecificpointintime.AnewChangeIdresults everytimesomeonecreatesasnapshot.Youshouldretainthisvaluewiththeversionofchangedatathat youextract(usingQueryChangedDiskAreas)fromthesnapshotsvirtualdisk. Whenyoubackupasnapshot,youeitherhaveapreviouslysavedChangeIdoryoudonot.Ifyouhavea savedChangeId,itidentifiesthelasttimeyouperformedabackup,andtellsthechangedblocktracking logictoidentifythosethathaveoccurredsincethetimeindicatedbytheChangeId.Ifyoudonothavea savedChangeId,thenyoumustsaveabaseline(full)backupofthevirtualdisk.Therearetwowaysto getthisbaselinebackup:(1)istodirectlysaveallofthevirtualdiskscontents,and(2)istoprovidethe specialChangeId*(star).ThestarChangeIdindicatesthatonlyactiveportionsofthevirtualdisk shouldbereturnedbyQueryChangedDiskAreas.Forboththinprovisioned(sparse)virtualdisksandfor ordinaryvirtualdisks,thisrepresentsasubstantialreductionintheamountofdatatobesaved.

Identifierforatimeinthepast.Thiscanbestar*toidentifyallallocatedareasofvirtualdisk,ignoring unallocatedareas(ofsparsedisk),oritcouldbeachangeIdstringsavedatthetimeofsnapshotbackup. ItonlymakessensetousethespecialChangeId=*whennopreviousChangeIdexists.Ifaprevious ChangeIddoesexist,thenQueryChangedDiskAreasreturnsthedisksectorsthatchangedsincethenew ChangeIdwascollected.Thefollowingtableshowsthealgorithm:

New Change ID change0 change1 Old Change ID none change0 Used for Query * change0 Result Allinusesectorsofthedisk. Allsectorsalteredsincechange0.

Enabling Changed Block Tracking

Thisfeatureisdisabledbydefault,becauseitreducesperformancebyasmallbutmeasurableamount.Ifyou querythevirtualmachineconfiguration,youcandetermineifitiscapableofChangedBlockTracking.Usethe propertycollectortoretrievethecapabilityfieldfromtheVirtualMachineManagedObject.Ifthecapability fieldcontainstheflagchangeTrackingSupported,thenyoucanproceed.Thevirtualmachineversionmust be7orhighertosupportthis.Ifthevirtualmachineversionislowerthan7,upgradethevirtualhardware. Ifsupported,youenableChangedBlockTrackingusinganabbreviatedformofVirtualMachineConfigSpec, thenusetheReconfigVM_TaskmethodtoreconfigurethevirtualmachinewithChangedBlockTracking:
VirtualMachineConfigSpec configSpec = new VirtualMachineConfigSpec; configSpec.SetChangeTrackingEnabled(new Boolean(true)); ManagedObjectReference taskMoRef = serviceConnection.getService().ReconfigureVm_Task(targetVM_MoRef, configSpec);

Poweredonvirtualmachinesmustgothroughastununstuncycle(poweron,resumeaftersuspend,migrate, orsnapshotcreate/delete/revert)beforethereconfigurationtakeseffect.

Copyright 2009 VMware, Inc. All rights reserved.

21

Designing Backup Solutions for VMware vSphere

To enable Changed Block Tracking with the VMware vSphere Client: 1 2 3 4 5 SelectthevirtualmachineandensurethatSummary>VMVersionsays7forvirtualhardwareversion. IntheSummarytab,clickEditSettings>Options>Advanced>General. Intherightsideofthedialogbox,clickConfigurationParameters... Inthenewdialogbox,locatetherowfornamectkEnabled,andchangeitsvaluefromfalsetotrue. Seeaboveconcerningthestununstuncycle.

To enable Changed Block Tracking and back up with the VMware vSphere API: 1 2 3 4 5 6 Querychangetrackingstatusofthevirtualmachine.Iffalse,activateChangedBlockTracking. Createasnapshotofthevirtualmachine.Thesnapshotoperationcausesastununstuncycle. StartingfromthesnapshotsConfigInfo,workyourwaytotheBackingInfoofallvirtualdisksinthe snapshot.ThisgivesyouthechangeIDsforallthedisksofthevirtualmachine. HoldontothechangeIDsanddoafullbackupofthesnapshot,sincethisisthefirsttimeforbackup. Deletethesnapshotwhenyourbackuphascompleted. Nexttimeyoubackupthisvirtualmachine,createasnapshotanduseQueryChangedDiskAreaswiththe changeIDsfromyourpreviousbackuptotakeadvantageofChangedBlockTracking.

Gathering Changed Block Information

AssociatedwithChangedBlockTrackingischangeId,anidentifierforversionsofchangedblockdata. Wheneveravirtualmachinesnapshotiscreated,associatedwiththatsnapshotisachangeIdthatfunctions asalandmarktoidentifychangesinvirtualdiskdata.Soitfollowsthatwhenasnapshotiscreatedforthe purposeofcreatinganinitialvirtualdiskbackup,thechangeIdassociatedwiththatsnapshotcanbeusedto retrievechangesthathaveoccurredsincesnapshotcreation. ToobtainthechangeIdassociatedwithanydiskinasnapshot,youexaminethehardwarearrayfromthe snapshot.Anyiteminthedevicestablethatisoftypevim.vm.device.VirtualDevice.VirtualDisk enclosesaclassdescribingthebackingstorage(obtainedusinggetBacking)thatimplementsvirtualdisk. Ifbackingstorageisoneofthefollowingtypes,youcanusethegetChangeIdmethodtoobtainthechangeId:
vim.vm.device.VirtualDevice.VirtualDisk.FlatVer2BackingInfo vim.vm.device.VirtualDevice.VirtualDisk.SparseVer2BackingInfo vim.vm.device.VirtualDevice.VirtualDisk.RawDiskVer2BackingInfo

InformationreturnedfromtheQueryChangedDiskAreasmethodisaVirtualMachine.DiskChangeInfo dataobjectcontainingadescriptionoftheareaofthediskcovered(startoffsetandlength),andanarrayof VirtualMachine.DiskChangeInfo.DiskChangeExtentitemsthatdescribethestartingoffsetandlengthof variousdiskareasthathavechanged. Forbestresults,whenusingQueryChangedDiskAreastogatherinformationaboutsnapshots,enablechange trackingbeforetakingsnapshots.Attemptstocollectinformationaboutchangestosnapshotsthatoccurred beforechangetrackingwasenabledresultinaFileFaulterror.Enablingchangetrackingprovidesthe additionalbenefitofsavingdiskspacebecauseitallowsforthebackupofonlyinformationthathaschanged. Ifchangetrackingisnotenabled,itisunknownwhathaschanged,sotheentirevirtualmachinemustbe backedupeachtime,ratherthanonlybackingupchangeddata. Beawarethatentiredisksareallocated,andthereforearereturnedwhenQueryChangedDiskAreasisusedin thefollowingscenarios:

Aclonedthickdiskiscreated. DisksareformattedusingtheLongFormatsetting. Disksarecreatedwithvmkfstoolsusingtheeagerzeroedthickformatoption. AVMDKisstoredonstoragethatdoesnotsupportVMFS,suchasNFSoriSCSIdevices.

Copyright 2009 VMware, Inc. All rights reserved.

22

Designing Backup Solutions for VMware vSphere

Tofindchangeinformation,youcanusetheManagedObjectBrowserathttp://<ESXhost>/mobtofollowpath content>hafolderroot>hadatacenter>datastore>vm>snapshot>config>hardware>virtualDisk. Changedblocktrackingisassociatedwiththesnapshot,notwiththebasedisk. Thefollowingcodesampleassumesthat,inthepast,youobtainedacompletecopyofthevirtualdisk,andat thetimewhenthechangeIdassociatedwiththesnapshotwascollected,youstoreditforuseatalatertime, whichisnow.Anewsnapshothasbeencreated,andtheappropriateMoRefisavailable:

String changeId; // Already initialized ManagedObjectReference theSnapshot; ManagedObjectReference theVM; int diskDeviceKey; VirtualMachine.DiskChangeInfo changes; long position = 0; do { changes = theVM.queryChangedDiskAreas(theSnapshot, diskDeviceKey, position, changeId); for (int i = 0; i < changes.changedArea.length; i++) { long length = changes.changedArea[i].length; long offset = changes.changedArea[i].startOffset; // // Go get and save disk data here } position = changes.startOffset + changes.length; } while (position < diskCapacity);

Intheabovecode,QueryChangedDiskAreasiscalledrepeatedly,aspositionmovesthroughthevirtualdisk. ThisisbecausethenumberofentriesintheChangedDiskAreaarraycouldendupoccupyingalargeamount ofmemoryfordescribingmanychangestoalargevirtualdisk. ThechangeId(changedblockID)containsasequencenumberintheform<UUID>/<nnn>.Ifthe<UUID> changes,itindicatesthattrackinginformationhasbecomeinvalid,necessitatingafullbackup.Otherwise incrementalbackupscancontinueintheusualpattern.

Troubleshooting
IfyoureconfigureavirtualmachinetosetchangeTrackingEnabled,butthepropertyremainsfalse,check thatyouhavequeriedthevirtualmachinestatuswithVirtualMachine->config()afterreconfiguration withVirtualMachine->reconfigure().Yes,youmustdoitagain.Alsomakesurethatthevirtualmachine ishardwareversion7andhasundergoneastununstuncyclesincereconfiguration.

Checking for Namespace

YoucanavoidusingthequeryChangedDiskAreasAPIonESX3.5/3.5i(andearlier)storagebyparsingXML filesforthenamespace.Forprepackagedmethodsthatdothis,seetheseSDKcodesamples:
Axis/java/com/vmware/samples/version/displaynewpropertieshost/DisplayNewPropertiesHostV25.java Axis/java/com/vmware/samples/version/getvirtualdiskfiles/GetVirtualDiskFilesV25.java DotNet/cs/DisplayNewProperties/DisplayNewPropertiesV25.cs DotNet/cs/GetVirtualDiskFiles/GetVirtualDiskFilesV25.cs

Limitations on Changed Block Tracking

ChangedBlockTrackingdoesnotworkinanyofthefollowingcases:

Virtualhardwareversionis6orearlier. ThevirtualdiskisaphysicalcompatibilityRDM. ThevirtualdiskisattachedtoasharedvirtualSCSIbus.

ChangedBlockTrackingcanbeenabledonvirtualmachinesthathavediskslikethese,butwhenqueriedfor theirchangeID,thesedisksalwaysreturnanemptystring.Soifyouhaveavirtualmachinewitharegular O/SdiskandapassthroughRDMasadatadiskyoucantrackchangesontheO/Sdiskonly.

Copyright 2009 VMware, Inc. All rights reserved.

23

Designing Backup Solutions for VMware vSphere

Incremental Restore of Backup Data

Atsomepointyoumightneedtorestoreavirtualdiskfromthebackupdatathatyougatheredasdescribed insectionChangedBlockTrackingonVirtualDisksonpage 21.Theessentialprocedureisasfollows: 1 2 Poweroffthevirtualmachine,ifpoweredon. UsingtheVirtualMachineConfigInfothatcorrespondstothelastknowngoodstateoftheguest operatingsystem,recreatethevirtualmachineasdescribedinsectionUsingVirtualMachineConfigInfo toCreateaVirtualMachineonpage 20. Completelyreloadthebasevirtualdiskusingthefullbackupthatstartedthemostrecentseriesof incrementalbackups. Createasnapshot,asachildofthebasevirtualdisk. Sequentiallyrestoretheincrementalbackupdata.Youcandothiseitherforwardsorbackwards.Ifyou workforwards,therestoremightwritesomesectorsmorethanonce.Ifyouworkbackwards,youmust keeptrackofwhichsectorswererestoredsoastoavoidrestoringthemagainfromolderdata. GetthechangeIDofthebackupthisisbeingrestored,andusethechangetrackingAPIstofindtheareas ofthediskthatneedtoberestored.Thisinformationmustbecached,becauseonceyoustartrestoringa virtualdisk,thechangetrackingmechanismwillfail. Restoreonlychangedareastothevirtualdisksreferredtobythesnapshot.Thisensuresthatyoudonot writethedatatotheredologcreatedbythesnapshot.Whenrestoringathinprovisioned(sparse)disk, usethestar*changeIDtoavoidwritingzeroestotheunallocatedblocks. RepeatStep 6andStep 7asnecessarybyapplyingincrementalbackupdatasetsinorder. Optionally,reverttothebasevirtualdisk,thuseliminatingthesnapshot.

3 4 5

8 9

Working with Microsoft Shadow Copy

MicrosoftShadowCopy,alsocalledVolumeSnapshotService(VSS),isaWindowsServerdatabackupfeature forcreatingconsistentpointintimecopiesofdata(calledshadowcopies). WhenVMwareToolsperformVSSquiescingwhilecreatingasnapshotinaWindowsguest,theygeneratea vss-manifest.zipfile.Thisfilecontainsthebackupcomponentdocument(BCD)andwritermanifests.The hostagentstoresthismanifestfileinthesnapshotDirofthevirtualmachine.Backupapplicationsshoulduse theHTTPprotocoltoobtainthevss-manifest.zipfilesotheycansaveit. YoucanaccessthisHTTPprotocoleitherthroughtheCopyDatastoreFile_TaskmethodinthevSphereAPI, orusingthevifscommandinthevCLI(inESX/ESXi4),formerlycalledtheRCLI. Restoremustbedoneusingthebackupapplicationsguestagent.VMwarevStorageAPIsforDataProtection providenohostagentsupportforthis.ApplicationsauthenticatingwithSSPImightnotworkrightbecause HTTPaccesswilldemandausernameandpassword,unlessthesessionwasrecentlyauthenticated. ForinformationaboutVSS,seetheMicrosoftTechNetarticle,HowVolumeShadowCopyServiceWorks.For informationaboutSecuritySupportProviderInterface(SSPI),seetheMSDNWebsite.

If you have comments about this documentation, submit your feedback to: docfeedback@vmware.com VMware, Inc. 3401 Hillview Ave., Palo Alto, CA 94304 www.vmware.com Copyright 2009 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents. VMware, the VMware boxes logo and design, Virtual SMP, and VMotion are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. Item: EN-000165-01 | Revised: 11/16/09

24

Designing Backup Solutions for VMware vSphere

Conclusion
ThistechnicalnoteintroducedtheVMwarevStorageAPIsforDataProtection,givinganoverviewofbackup andrestoreoperationsinVMwarevSphere.Theinformationpresentedhereshouldhelpyouunderstandthe underlyingconceptsandstepstotakeforimplementingapplicationstoprotectvirtualmachinedata.

Copyright 2009 VMware, Inc. All rights reserved.

25