Summary of the content on the page No. 1
Tru64 UNIX
Kernel Debugging
Part Number: AA-RH99A-TE
July 1999
Product Version: Tru64UNIX Version 5.0or higher
Thismanualexplainshowtousetoolstodebugakernelandanalyzea
crashdumpoftheTru64UNIX(formerlyDIGITALUNIX)operating
system.Also,thismanualexplainshowtowriteextensionstothekernel
debuggingtools.
Compaq Computer Corporation
Houston, Texas
Summary of the content on the page No. 2
©1999CompaqComputerCorporation COMPAQandtheCompaqlogoRegisteredinU.S.PatentandTrademarkOffice. AlphaandTru64are trademarksofCompaqInformationTechnologiesGroup,L.PintheUnitedStatesandothercountries. MicrosoftandWindowsaretrademarksofMicrosoftCorporationintheUnitedStatesandothercountries. UNIXandTheOpenGrouparetrademarksofTheOpenGroup.Allotherproductnamesmentionedherein maybetrademarksoftheirrespectivecompanies. Confidentialcomputersoftware. ValidlicensefromCompaqrequiredforpossession,use,orcopyin
Summary of the content on the page No. 3
Contents About This Manual 1 Introduction to Kernel Debugging 1.1 LinkingaKernelImageforDebugging............................ 1–1 1.2 DebuggingKernelPrograms ........................................ 1–3 1.3 DebuggingtheRunningKernel..................................... 1–3 1.4 AnalyzingaCrashDumpFile ...................................... 1–5 2 Kernel Debugging Utilities 2.1 ThedbxDebugger .................................................... 2–2 2.1.1 InvokingthedbxDebuggerforKernelDebugging ...
Summary of the content on the page No. 4
2.2.3.9 DisassemblingInstructions ................................ 2–21 2.2.3.10 DisplayingRemoteExportedEntries .................... 2–21 2.2.3.11 DisplayingtheFileTable .................................. 2–21 2.2.3.12 DisplayingtheudbandtcbTables........................ 2–22 2.2.3.13 PerformingCommandsonLists .......................... 2–22 2.2.3.14 DisplayingthelockstatsStructures ...................... 2–24 2.2.3.15 DisplayinglockinfoStructures ............................ 2–25 2.2.3.16 Disp
Summary of the content on the page No. 5
3.2.7 CheckingArgumentsPassedtoanExtension ............... 3–7 3.2.8 CheckingtheFieldsinaStructure............................ 3–7 3.2.9 SettingthekdbxContext ....................................... 3–8 3.2.10 PassingCommandstothedbxDebugger..................... 3–9 3.2.11 DereferencingaPointer ......................................... 3–9 3.2.12 DisplayingtheErrorMessagesStoredinFields ............ 3–10 3.2.13 ConvertingaLongAddresstoaStringAddress ............ 3–10 3.2.14 FreeingMemory .......
Summary of the content on the page No. 6
A Output from the crashdc Command Index Examples 3–1 TemplateExtensionUsingLists.................................... 3–23 3–2 ExtensionThatUsesLinkedLists:callout.c ..................... 3–24 3–3 TemplateExtensionsUsingArrays ................................ 3–27 3–4 ExtensionThatUsesArrays:file.c................................. 3–28 3–5 ExtensionThatUsesGlobalSymbols:sum.c..................... 3–34 Figures 2–1 UsingaGatewaySystemDuringRemoteDebugging........... 2–38 Tables 2–1 ThedbxAddressMod
Summary of the content on the page No. 7
About This Manual Thismanualprovidesinformationonthetoolsusedtodebugakerneland analyzeacrashdumpfileoftheTru64™UNIX(formerlyDIGITALUNIX) operatingsystem. Italsoexplainshowtowriteextensionstothekernel debuggingtools. Youcanuseextensionstodisplaycustomizedinformation fromkerneldatastructuresoracrashdumpfile. Audience Thismanualisintendedforsystemprogrammerswhowriteprogramsthat usekerneldatastructuresandarebuiltintothekernel. Itisalsointended forsystemadministratorswhoareresponsibleformanagingtheop
Summary of the content on the page No. 8
Organization Thismanualconsistsoffourchaptersandoneappendix: Chapter1 Introduces the concepts of kernel debugging and crash dump analysis. Describes the tools used to debug kernels and Chapter2 analyze crash dump files. Chapter3 Describeshowtowritea kdbxdebuggerextension. This chapterassumesyouhavepurchasedandinstalledaTru64 UNIXSourceKitandsohaveaccesstosourcefiles. Chapter4 Providesbackgroundinformationusefulforandexamples of analyzing crashdump files. AppendixA Containsexampleoutputfromthe cr
Summary of the content on the page No. 9
G Manuals for general users S Manualsforsystemandnetworkadministrators P Manuals for programmers R Manualsforreferencepage users Somemanualsinthedocumentationhelpmeettheneedsofseveral audiences. Forexample,theinformationinsomesystemmanualsisalso usedbyprogrammers. Keepthisinmindwhensearchingforinformation onspecifictopics. TheDocumentationOverviewprovidesinformationonallofthemanualsin theTru64UNIXdocumentationset. Reader’s Comments Compaqwelcomesanycommentsandsuggestionsyouhaveonthisand otherTru
Summary of the content on the page No. 10
% $ ApercentsignrepresentstheCshellsystemprompt. Adollarsignrepresentsthesystempromptforthe Bourne,Korn,andPOSIXshells. # Anumbersignrepresentsthesuperuserprompt. % cat Boldfacetypeininteractiveexamplesindicates typeduserinput. file Italic(slanted)typeindicatesvariablevalues, placeholders,andfunctionargumentnames. [|] {|} Insyntaxdefinitions,bracketsindicateitemsthat areoptionalandbracesindicateitemsthatare required. Verticalbarsseparatingitemsinside bracketsorbracesindicatethatyouchooseoneitem
Summary of the content on the page No. 11
1 Introduction to Kernel Debugging Kerneldebuggingisatasknormallyperformedbysystemsengineerswriting kernelprograms. Akernelprogramisonethatisbuiltaspartofthekernel andthatreferenceskerneldatastructures. Systemadministratorsmight alsodebugthekernelinthefollowingsituations: • Aprocessishungorstopsrunningunexpectedly • Theneedarisestoexamine,andpossiblymodify,kernelparameters • Thesystemitselfhangs,panics,orcrashes Thismanualdescribeshowtodebugkernelprogramsandthekernel.Italso includesinformationab
Summary of the content on the page No. 12
textfilethatdescribesthehardwareandsoftwarethatwillbepresenton therunningsystem. Usingthisinformation,thebootstraplinkerlinksthe modulesthatareneededtosupportthishardwareandsoftware.Thelinker buildsthekerneldirectlyintomemory. Youcannotdirectlydebugabootstrap-linkedkernelbecauseyoumustsupply thenameofanimagetothekerneldebuggingtools.Withouttheimage,the toolshavenoaccesstosymbolnames,variablenames,andsoon.Therefore, thefirststepinanykerneldebuggingeffortistodeterminewhetheryour kernelwaslinkedatb
Summary of the content on the page No. 13
1.2 Debugging Kernel Programs Kernelprogramscanbedifficulttodebugbecauseyounormallycannot controlkernelexecution. Tomakedebuggingkernelprogramsmore convenient,thesystemprovidesthe kdebugdebugger. The kdebug debuggeriscodethatresidesinsidethekernelandallowsyoutousethedbx debuggertocontrolexecutionofarunningkernelinthesamemanneras youcontrolexecutionofauserspaceprogram. Todebugakernelprogram inthismanner,followthesesteps: 1. Buildyourkernelprogramintothekernelonatestsystem. 2. Setupthe kdebugdebug
Summary of the content on the page No. 14
# dbx -k /vmunix /dev/mem Thiscommandinvokes dbxwiththekerneldebuggingflag, −k,which mapskerneladdressestomakekerneldebuggingeasier.The/vmunixand /dev/memparameterscausethedebuggertooperateontherunningkernel. Onceinthedbxenvironment,youusedbxcommandstodisplayprocessIDs andtraceexecutionofprocesses.Youcanperformthesametasksusingthe kdbxdebugger. Thefollowingexampleshowsthedbxcommandyouuseto displayprocessIDs: (dbx) kps PID COMM 00000 kernel idle 00001 init 00014 kloadsrv 00016 update . . . Ifyouw
Summary of the content on the page No. 15
1.4 Analyzing a Crash Dump File Ifyoursystemcrashes,youcanoftenfindthecauseofthecrashbyusing dbxorkdbxtodebugoranalyzeacrashdumpfile. Theoperatingsystemcancrashbecauseoneofthefollowingoccurs: • Hardwareexception • Softwarepanic • Hungsystem Whenasystemhangs,itisoftennecessarytoforcethesystemtocreate dumpsthatyoucananalyzetodeterminewhythesystemhung. The SystemAdministrationmanualdescribestheprocedureforforcinga crashdumpofahungsystem. • Resourceexhaustion Thesystemcrashesorhangsbecauseitcannotco
Summary of the content on the page No. 16
Atsystemreboottime,thecopyofcorememorysavedintheswappartitions iscopiedintoafile,calledacrashdumpfile. Youcananalyzethecrash dumpfiletodeterminewhatcausedthecrash.Bydefault,thecrashdumpis apartial(ratherthanfull)dumpandisincompressedform. Forcomplete informationaboutmanagingcrashdumpsandcrashdumpfiles,including howtochangedefaultsettings,seetheSystemAdministrationmanual.For examplesofanalyzingcrashdumpfiles,seeChapter4. 1–6 Introduction to Kernel Debugging
Summary of the content on the page No. 17
2 Kernel Debugging Utilities TheTru64UNIXsystemprovidesseveraltoolsyoucanusetodebugthe kernelandkernelprograms.TheLadebugdebugger(availableasanoption) isalsocapableofdebuggingthekernel. Thischapterdescribesthreedebuggersandautilityforanalyzingcrash dumps: • Thedbxdebugger,whichisdescribedforkerneldebugginginSection2.1. (Forgeneraldbxuserinformation,seetheProgrammer’sGuide.) Youcanusethedbxdebuggertodisplaythevaluesofkernelvariables andkernelstructures. However,youmustunderstandthestructures andb
Summary of the content on the page No. 18
______________________ Note _______________________ StartingwithTru64UNIXVersion5.0,allthepreviously mentionedtoolscanbeusedwithcompressed(vmzcore.n)and uncompressed(vmcore.n)crashdumpfiles. Olderversionsof thesetoolscanreadonly vmcore.nfiles. Ifyouareusingan olderversionofatool,usetheexpand_dumputilitytoproduce a vmcore.nfilefroma vmzcore.nfile. Formoreinformation aboutcompressedanduncompressedcrashdumpfiles,see expand_dump(8)andtheSystemAdministrationmanual. 2.1 The dbx Debugger The dbxdebugge
Summary of the content on the page No. 19
• Thesystemcorememoryimage Thesefilesmaybefilesfromarunningsystem,suchas /vmunixand /dev/mem,ordumpfiles,suchasvmunix.nandvmzcore.n(compressed) orvmcore.n(uncompressed). Bydefault,crashdumpfilesarecreatedin the/var/adm/crashdirectory(seetheSystemAdministrationmanual). ______________________ Note _______________________ Youmightneedtobethesuperuser(rootlogin)toexaminethe runningsystemorcrashdumpfilesproducedby savecore. Whetheryouneedtobethesuperuserdependsonthedirectory andfileprotectionsforthef
Summary of the content on the page No. 20
wanttoaddasymboltabletoyourcurrentdebuggingsessionratherthan endthesessionandstartanewone. Toaddasymboltabletoyourcurrent debuggingsession,followthesesteps: 1. Gotoawindowotherthantheoneinwhichthedebuggerisrunning, orputthedebuggerinthebackground,andrebuildthemodulesfor whichyouneedasymboltable. 2. Oncethemodulesbuildcorrectly,usetheostripcommandtostripa symboltableoutoftheresultingexecutablefile. Forexample,ifyour executablefileisnamed kernel_program,issueacommandsuchas thefollowingone: % /usr/
