05/09/2016

AnindepthlookatGhostRecordsinSQLServerSQLJourney

AnindepthlookatGhostRecordsinSQLServerSQL
Journey
GhostrecordsaresomethingthatareabitofanenigmaformostfolksworkingwithSQLServer,andnotjust
becauseofthename.Today,Illseektoexplaintheconcept,aswellasidentifysometroubleshooting
techniques.
ThemainreasonbehindintroducingtheconceptofGhostrecordswastoenhanceperformance.Intheleaf
levelofanindex,whenrowsaredeleted,theyremarkedasghostrecords.Thismeansthattherowstayson
thepagebutabitischangedintherowheadertoindicatethattherowisreallyaghost.Thepageheaderalso
reflectsthenumberofghostrecordsonapage.Whatthismeans,ineffect,isthattheDMLoperationwhich
firedthedeletewillreturntotheusermuchfaster,becauseitdoesnothavetowaitfortherecordstobedeleted
physically.Rather,theyrejustmarkedasghosted.Ghostrecordsarepresentonlyintheindexleafnodes.If
ghostrecordswerentused,theentirerangesurroundingadeletedkeywouldhavetobelocked.Heresan
exampleipickedupfromsomewhere:
Supposeyouhaveauniqueindexonanintegerandtheindexcontainsthevalues1,30,and100.Ifyoudelete
30,SQLServerwillneedtolock(andpreventinsertsinto)theentirerangebetween1and100.Withghosted
records,the30isstillvisibletobeusedasanendpointofakeyrangelocksothatduringthedeletetransaction,
SQLServercanallowinsertsforanyvalueotherthan30toproceed.SQLServerprovidesaspecial
housekeepingthreadthatperiodicallychecksBtreesforghostedrecordsandasynchronouslyremovesthem
fromtheleafleveloftheindex.Thissamethreadcarriesouttheautomaticshrinkingofdatabasesifyouhave
thatoptionset.Theghostrecord(s)presenceisregisteredin:
Therecorditself
ThePageonwhichtherecordhasbeenghosted
ThePFSforthatpage(fordetailsonPFS,seePaulRandalsbloghere)
TheDBTABLEstructureforthecorrespondingdatabase.YoucanviewtheDBTABLEstructurebyusingthe
DBCCDBTABLEcommand(makesureyouhaveTF3604turnedon).
Theghostrecordscanbecleanedupin3ways:
Ifarecordofthesamekeyvalueasthedeletedrecordisinserted
Ifthepageneedstobesplit,theghostrecordswillbehandled
TheGhostcleanuptask(scheduledtorunonceevery5seconds)
TheGhostcleanupprocessdividestheghostpagesinto2categories:
HotPages(frequentlyvisitedbyscanningprocesses)
ColdPages
TheGhostcleanupthreadisabletoretrievethelistofColdpagesfromtheDBTABLEforthatdatabase,orthe
PFSPageforthatinterval.Thecleanuptaskcleansupamaximumof10ghostpagesatatime.Also,while
searchingfortheghostpages,ifitcovers10PFSPages,ityields.Asfarashotghostpagesareconcerned,the
ghostcleanupstrivestokeepthenumberofsuchpagesbelowaspecifiedlimit.Also,ifthethreadcleansup10
https://blogs.msdn.microsoft.com/sqljourney/2012/07/27/anindepthlookatghostrecordsinsqlserver/#comment6505

1/3

05/09/2016

AnindepthlookatGhostRecordsinSQLServerSQLJourney

hotghostpages,ityields.However,ifthenumberofhotghostpagesisabovethespecified(hardcoded)limit,
thetaskrunsnonstoptillthecountcomesdownbelowthethresholdvalue.IfthereisnoCPUusageonthe
system,theGhostcleanuptaskrunstilltherearenomoreghostpagestocleanup.TroubleshootingSonowwe
gettotheinterestingpart.Ifyoursystemhassomehugedeleteoperations,andyoufeelthespaceisnotbeing
freedupatallorevennotattherateitshouldbe,youmightwanttocheckifthereareghostrecordsinthat
database.Illtrytobreakdownthetroubleshootingintosomelogicalstepshere:
1.Runthefollowingcommand:
Select*fromsys.dm_db_index_physical_stats(db_id(<dbname>),<ObjectID>,NULL,NULL,DETAILED)
P.S.TheobjectIDcanbelookedupfromsys.objectsbyfilteringonthenamecolumn.
2.ChecktheGhost_Record_CountandVersion_Ghost_Record_Countcolumns(versionghostrecordcount
willbepopulatedwhenyoureusingsnapshotisolationonthedatabase).Ifthisishigh(severalmillionin
somecases),thenyouvemostprobablygotaghostrecordcleanupissue.IfthisisSQLServer2008/2008
R2,thenmakesureyouhaveappliedthepatchmentionedinthekb
http://support.microsoft.com/kb/2622823
3.Tryrunningthefollowingcommand:
EXECsp_clean_db_free_space@dbname=N<dbname>
4.Iftheghostrecordcountfromstep1isthesame(orsimilar)afterrunningthiscommand,thenwemight
needtodiginabitdeeper.
Warning:Someofthetroubleshootingstepsmentionedfromhereonareunpublishedandmightbe
unsupportedbyMicrosoft.Proceedatyourownrisk.
5.EnableTraceFlag662(printsdetailedinformationabouttheworkdonebytheghostcleanuptaskwhenit
runsnext),and3605(directstheoutputofTF662totheSQLerrorlog).Pleasedothisduringoffhours.
6.Waitforafewminutes,thenexaminetheerrorlog.First,youneedtocheckifthedatabaseisbeingtouched
atall.Ifso,itsverymuchpossiblethattheGhostCleanuptaskisdoingitsjob,andwillprobablycatchupin
abit.Anotherthingtowatchoutforis,doyouseeonepagebeingcleanedupmultipletimes?Ifso,notethe
pagenumberandfileid.PleaseensureyoudisabletheTF662afterthisstep(itcreatesalotofnoiseinthe
errorlog,sopleaseuseitforaslittletimeaspossible)
7.Next,runthefollowingcommandonthepagetoviewitscontents
DBCCPAGE(<DBName>,<fileid>,<Pageno.>,3)
8.Thiswillgiveyouthecontentsofthepage.seeifyoucanspotafieldcalledm_ghostRecCntintheoutput.If
ithasanonzerovalue,thanmeansthepagehasghostrecords.Also,lookforthePFSpageforthatpage.It
willlooksomethinglikePFS(1:1).YoucanalsotrydumpingthePFSpagetoseeifthispagehasaHas
Ghostagainstit.FormoredetailsontheDBCCPage,checkoutPaulRandalsposthere
AnotherthingthatdeservesmentionisthespecialroleofthePAGLOCKhintw.r.tghostrecords:
https://blogs.msdn.microsoft.com/sqljourney/2012/07/27/anindepthlookatghostrecordsinsqlserver/#comment6505

2/3

05/09/2016

AnindepthlookatGhostRecordsinSQLServerSQLJourney

RunningaselectstatementwiththePAGLOCKhintagainstatablewillensurethatalltheghostrecordsin
thattablearequeuedforcleanupbytheghostcleanuptask.
AccommodatingthePAGLOCKhintinyourdeletestatementwillensurethattherecordsaredeletedthere
andthen,andarenotleftbehindfortheGhostCleanuptasktotakecareoflater.Bydefault,allindexeshave
thePAGLOCKoptionturnedon(youcancheckbyscriptingoutacreateindextask),buttheymightnotbe
abletogetitallthetime.ThisiswherethePAGLOCKqueryhintcomesin.Itmakesyourquerywaitforthe
PageLock,soitcancleanuptherecordsphysicallybeforereturning.However,itsnotadvisabletousethe
PAGLOCKhintinyourdeletestatementsallthetime,astheperformancetradeoffalsoneedstobetaken
intoconsideration(thisisthesamepurposeforwhichtheGhostCleanuptaskwasintroduced,remember?).
ThisshouldberesortedtoonlyundersituationswhereyouarefacingadefiniteissuewithGhostRecord
cleanup,andhaveadireneedtopreventfurtherghostrecordsfromgettingcreated.
Thesestepsmightormightnotsolveyourproblem,butwhattheywilldoisgiveyouaninsightintohowtheSQL
ServerDatabaseEngineworksw.r.tGhostrecordsandtheircleanup.Oneofthemostcommon(and
quickest)resolutionsforaghostrecordsissueistorestartSQLServer.Onceagain,thispostdoesnot
comewithanyguarantees,andthecontentsareinnowayendorsedbyMicrosoftoranyothercorporationor
individual.HopethishelpsyouunderstandtheconceptofGhostRecordssomewhat.Youremorethanwelcome
toshareyourexperiences/opinions/knowledgeinthecommentssection,andIshallbedelightedtoinclude
theminthecontentsofthepostifsuitable.

https://blogs.msdn.microsoft.com/sqljourney/2012/07/27/anindepthlookatghostrecordsinsqlserver/#comment6505

3/3