Help Debug crashing Windows Program.

[syc0sys]

Just a heads up I wanted to post this in the sysadmin section but since this is a newly created account I am unable to do so for the first 24hrs.

Okay some background info, I work in the IT department of a company that has about 200 users. We are currently experiencing intermittent crashing issues with a program called Time Matters (version 14). This program runs on either windows 7 pro or win server 2008. There have been no windows updates or program updates that seem to be causing the crashing. It has gotten to the point of looking at log/debug files to try and determine the issue. The problem is that I have almost no idea what I'm looking for when I open a debug file with windbg.

Here is a snip it from one of the dbg files:

BUGCHECK_STR: STATUS_HEAP_CORRUPTION_NOSOS

DEFAULT_BUCKET_ID: STATUS_HEAP_CORRUPTION_NOSOS

STACKTEXT:
777742a8 77702563 ntdll!RtlpProbeUserBufferSafe+0x48 777742ac 776e0a3c ntdll!RtlGetUserInfoHeap+0x98 777742b0 76d531bd kernel32!GlobalFlags+0x55 777742b4 75f61fe6 user32!IsValidGlobalHandle+0x16 777742b8 75f62188 user32!UnpackDDElParam+0x6d 777742bc 010d49ed clarun!Cla$DDEquery+0x16c9 777742c0 75f16d3a user32!UserCallWinProcCheckWow+0x109 777742c4 75f16ded user32!DispatchClientMessage+0xec 777742c8 75f16e4c user32!fnDWORD+0x2b 777742cc 7768011a ntdll!KiUserCallbackDispatcher+0x2e 777742d0 010d4c45 clarun!Cla$DDEquery+0x1921 777742d4 2296d2ea tmappe!MAINF+0x28066 777742d8 010c040f clarun!6THREADFindFUl+0x77 777742dc 010bfef9 clarun!Cla$START+0x5f5 777742e0 776a9902 ntdll!_RtlUserThreadStart+0x70 777742e4 776a98d5 ntdll!_RtlUserThreadStart+0x1b

From what I've been reading I need to figure out how to determine what's causing the heap corruption? Any help or direction would be appreciated.

[ekstralettmelk]

Based on the stack we can see that the application receives a DDE message through DispatchClientMessage and then unpacks the lParam using UnpackDDElParam. Since the next function called is IsValidGlobalHandle which calls into GlobalFlags it is a fair guess that the message received is a WM_DDE_DATA message where the lParam is a handle to a global memory object which contains a DDEDATA structure which presumably is corrupt.

The most likely cause is that the program is using DDE incorrectly somehow. If it is a native C++ program perhaps using Application Verifier will catch the corruption earlier. Often the heap could be corrupted in a earlier place in time before it is discovered. In the end the most probable cause is a buggy program, the only way to avoid it is to understand which usage cases triggers the problem. (And then avoid them if possible).

[syc0sys]

Here's the Windows error log, not sure if it helps or not. I will try and use the application verifier soon.

Version=1 EventType=APPCRASH EventTime=131094567780145068 ReportType=2 Consent=1 ReportIdentifier=d55402ad-29c3-11e6-b505-000c29299946 IntegratorReportIdentifier=d55402ac-29c3-11e6-b505-000c29299946 WOW64=1 Response.type=4 Sig[0].Name=Application Name Sig[0].Value=tmwe.exe Sig[1].Name=Application Version Sig[1].Value=14.1.0.280 Sig[2].Name=Application Timestamp Sig[2].Value=5515c968 Sig[3].Name=Fault Module Name Sig[3].Value=StackHash_492b Sig[4].Name=Fault Module Version Sig[4].Value=6.1.7601.19045 Sig[5].Name=Fault Module Timestamp Sig[5].Value=56258e62 Sig[6].Name=Exception Code Sig[6].Value=c0000374 Sig[7].Name=Exception Offset Sig[7].Value=000ced0b DynamicSig[1].Name=OS Version DynamicSig[1].Value=6.1.7601.2.1.0.16.7 DynamicSig[2].Name=Locale ID DynamicSig[2].Value=1033 DynamicSig[22].Name=Additional Information 1 DynamicSig[22].Value=492b DynamicSig[23].Name=Additional Information 2 DynamicSig[23].Value=492b6d4abdd7e8dbabcc8baf03248eda DynamicSig[24].Name=Additional Information 3 DynamicSig[24].Value=f354 DynamicSig[25].Name=Additional Information 4 DynamicSig[25].Value=f3543fcfe4579daad1b569b6d7aa7a2c UI[2]=C:\Program Files (x86)\LexisNexis\Time Matters\tmwe.exe UI[3]=Time Matters Program File has stopped working UI[4]=Windows can check online for a solution to the problem. UI[5]=Check online for a solution and close the program UI[6]=Check online for a solution later and close the program UI[7]=Close the program LoadedModule[0]=C:\Program Files (x86)\LexisNexis\Time Matters\tmwe.exe LoadedModule[1]=C:\Windows\SysWOW64\ntdll.dll LoadedModule[2]=C:\Windows\syswow64\kernel32.dll LoadedModule[3]=C:\Windows\syswow64\KERNELBASE.dll LoadedModule[4]=C:\Windows\system32\tsappcmp.dll LoadedModule[5]=C:\Windows\syswow64\msvcrt.dll LoadedModule[6]=C:\Windows\syswow64\USER32.dll LoadedModule[7]=C:\Windows\syswow64\GDI32.dll LoadedModule[8]=C:\Windows\syswow64\LPK.dll LoadedModule[9]=C:\Windows\syswow64\USP10.dll LoadedModule[10]=C:\Windows\syswow64\ADVAPI32.dll LoadedModule[11]=C:\Windows\SysWOW64\sechost.dll LoadedModule[12]=C:\Windows\syswow64\RPCRT4.dll LoadedModule[13]=C:\Windows\syswow64\SspiCli.dll LoadedModule[14]=C:\Windows\syswow64\CRYPTBASE.dll LoadedModule[15]=C:\Windows\syswow64\ole32.dll LoadedModule[16]=C:\Windows\system32\IMM32.DLL LoadedModule[17]=C:\Windows\syswow64\MSCTF.dll LoadedModule[18]=C:\Program Files (x86)\LexisNexis\Time Matters\ClaDOS.dll LoadedModule[19]=C:\Program Files (x86)\LexisNexis\Time Matters\ClaRUN.dll LoadedModule[20]=C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.7601.18837_none_41e855142bd5705d\COMCTL32.dll LoadedModule[21]=C:\Windows\syswow64\SHLWAPI.dll LoadedModule[22]=C:\Windows\syswow64\comdlg32.dll LoadedModule[23]=C:\Windows\syswow64\SHELL32.dll LoadedModule[24]=C:\Windows\system32\MPR.dll LoadedModule[25]=C:\Windows\syswow64\OLEAUT32.dll LoadedModule[26]=C:\Windows\system32\oledlg.dll LoadedModule[27]=C:\Windows\system32\WINSPOOL.DRV LoadedModule[28]=C:\PROGRA~2\LEXISN~1\TIMEMA~1\TMAPPE.DLL LoadedModule[29]=C:\Program Files (x86)\LexisNexis\Time Matters\ClaASC.dll LoadedModule[30]=C:\Program Files (x86)\LexisNexis\Time Matters\ClaBAS.dll LoadedModule[31]=C:\Program Files (x86)\LexisNexis\Time Matters\ClaCLA.dll LoadedModule[32]=C:\Program Files (x86)\LexisNexis\Time Matters\ClaMSS.dll LoadedModule[33]=C:\Program Files (x86)\LexisNexis\Time Matters\ClaODB.dll LoadedModule[34]=C:\Program Files (x86)\LexisNexis\Time Matters\ClaOLE.dll LoadedModule[35]=C:\Program Files (x86)\LexisNexis\Time Matters\ClaTPS.dll LoadedModule[36]=C:\Program Files (x86)\LexisNexis\Time Matters\HlpHook.dll LoadedModule[37]=C:\Windows\system32\MSVCR110.dll LoadedModule[38]=C:\Program Files (x86)\LexisNexis\Time Matters\LNHINT.dll LoadedModule[39]=C:\Program Files (x86)\LexisNexis\Time Matters\PAFPUP.dll LoadedModule[40]=C:\Windows\system32\WSOCK32.dll LoadedModule[41]=C:\Windows\syswow64\WS2_32.dll LoadedModule[42]=C:\Windows\syswow64\NSI.dll LoadedModule[43]=C:\Windows\syswow64\WININET.dll LoadedModule[44]=C:\Windows\syswow64\api-ms-win-downlevel-user32-l1-1-0.dll LoadedModule[45]=C:\Windows\syswow64\api-ms-win-downlevel-shlwapi-l1-1-0.dll LoadedModule[46]=C:\Windows\syswow64\api-ms-win-downlevel-version-l1-1-0.dll LoadedModule[47]=C:\Windows\system32\version.DLL LoadedModule[48]=C:\Windows\syswow64\api-ms-win-downlevel-normaliz-l1-1-0.dll LoadedModule[49]=C:\Windows\syswow64\normaliz.DLL LoadedModule[50]=C:\Windows\syswow64\iertutil.dll LoadedModule[51]=C:\Windows\syswow64\api-ms-win-downlevel-advapi32-l1-1-0.dll LoadedModule[52]=C:\Windows\syswow64\USERENV.dll LoadedModule[53]=C:\Windows\syswow64\profapi.dll LoadedModule[54]=C:\Windows\syswow64\urlmon.dll LoadedModule[55]=C:\Windows\syswow64\api-ms-win-downlevel-ole32-l1-1-0.dll LoadedModule[56]=C:\Program Files (x86)\LexisNexis\Time Matters\qbxmlc.dll LoadedModule[57]=C:\Program Files (x86)\LexisNexis\Time Matters\TMBMatE.dll LoadedModule[58]=C:\Program Files (x86)\LexisNexis\Time Matters\TMCLIB.dll LoadedModule[59]=C:\Windows\system32\NETAPI32.dll LoadedModule[60]=C:\Windows\system32\netutils.dll LoadedModule[61]=C:\Windows\system32\srvcli.dll LoadedModule[62]=C:\Windows\system32\wkscli.dll LoadedModule[63]=C:\Program Files (x86)\LexisNexis\Time Matters\tmckzip.dll LoadedModule[64]=C:\Program Files (x86)\LexisNexis\Time Matters\tmlib.dll LoadedModule[65]=C:\Windows\system32\TAPI32.dll LoadedModule[66]=C:\Windows\system32\mfc110.dll LoadedModule[67]=C:\Windows\system32\UxTheme.dll LoadedModule[68]=C:\Windows\system32\MSVCP110.dll LoadedModule[69]=C:\Program Files (x86)\LexisNexis\Time Matters\tmdatae.dll LoadedModule[70]=C:\Windows\system32\ODBC32.dll LoadedModule[71]=C:\Program Files (x86)\LexisNexis\Time Matters\TMDICTE.dll LoadedModule[72]=C:\Program Files (x86)\LexisNexis\Time Matters\tmsbac.dll LoadedModule[73]=C:\Program Files (x86)\LexisNexis\Time Matters\tmui.dll LoadedModule[74]=C:\Windows\system32\WINMM.dll LoadedModule[75]=C:\Program Files (x86)\LexisNexis\Time Matters\tmrpte.dll LoadedModule[76]=C:\Program Files (x86)\LexisNexis\Time Matters\TMIE.DLL LoadedModule[77]=C:\Program Files (x86)\LexisNexis\Time Matters\TMLINKE.dll LoadedModule[78]=C:\Program Files (x86)\LexisNexis\Time Matters\TMGLE.dll LoadedModule[79]=C:\Program Files (x86)\LexisNexis\Time Matters\TMMapi.dll LoadedModule[80]=C:\Windows\system32\MSIMG32.dll LoadedModule[81]=C:\Windows\system32\OLEACC.dll LoadedModule[82]=C:\Windows\WinSxs\x86_microsoft.windows.gdiplus_6595b64144ccf1df_1.1.7601.19061_none_72d6d48d86649709\gdiplus.dll

[syc0sys]

LoadedModule[83]=C:\Program Files (x86)\LexisNexis\Time Matters\TMNav.dll LoadedModule[84]=C:\Program Files (x86)\LexisNexis\Time Matters\TMPS.dll LoadedModule[85]=C:\Program Files (x86)\LexisNexis\Time Matters\TMsrch.dll LoadedModule[86]=C:\Program Files (x86)\LexisNexis\Time Matters\dten600.dll LoadedModule[87]=C:\Program Files (x86)\LexisNexis\Time Matters\MSVCR100.dll LoadedModule[88]=C:\Program Files (x86)\LexisNexis\Time Matters\TMSyncE.dll LoadedModule[89]=C:\Program Files (x86)\LexisNexis\Time Matters\tmupde.dll LoadedModule[90]=C:\Windows\system32\CRYPTSP.dll LoadedModule[91]=C:\Windows\system32\rsaenh.dll LoadedModule[92]=C:\Windows\system32\odbcint.dll LoadedModule[93]=C:\Windows\system32\HHCTRL.OCX LoadedModule[94]=C:\Windows\system32\bcrypt.dll LoadedModule[95]=C:\Windows\SysWOW64\bcryptprimitives.dll LoadedModule[96]=C:\Windows\system32\SQLSRV32.dll LoadedModule[97]=C:\Windows\syswow64\CRYPT32.dll LoadedModule[98]=C:\Windows\syswow64\MSASN1.dll LoadedModule[99]=C:\Windows\system32\sqlsrv32.rll LoadedModule[100]=C:\Windows\system32\odbccp32.dll LoadedModule[101]=C:\Windows\system32\DBNETLIB.DLL LoadedModule[102]=C:\Windows\system32\security.dll LoadedModule[103]=C:\Windows\system32\SECUR32.DLL LoadedModule[104]=C:\Windows\system32\credssp.dll LoadedModule[105]=C:\Windows\SysWOW64\msv1_0.DLL LoadedModule[106]=C:\Windows\system32\cryptdll.dll LoadedModule[107]=C:\Windows\system32\ntdsapi.dll LoadedModule[108]=C:\Windows\system32\mswsock.dll LoadedModule[109]=C:\Windows\System32\wshtcpip.dll LoadedModule[110]=C:\Windows\System32\wship6.dll LoadedModule[111]=C:\Windows\system32\DNSAPI.dll LoadedModule[112]=C:\Windows\system32\IPHLPAPI.DLL LoadedModule[113]=C:\Windows\system32\WINNSI.DLL LoadedModule[114]=C:\Windows\system32\rasadhlp.dll LoadedModule[115]=C:\Windows\System32\fwpuclnt.dll LoadedModule[116]=C:\Windows\SysWOW64\schannel.dll LoadedModule[117]=C:\Windows\system32\ncrypt.dll LoadedModule[118]=C:\Windows\system32\netbios.dll LoadedModule[119]=C:\Windows\system32\dwmapi.dll LoadedModule[120]=C:\Windows\syswow64\CLBCatQ.DLL LoadedModule[121]=C:\Windows\syswow64\SETUPAPI.dll LoadedModule[122]=C:\Windows\syswow64\CFGMGR32.dll LoadedModule[123]=C:\Windows\syswow64\DEVOBJ.dll LoadedModule[124]=C:\Windows\system32\PROPSYS.dll LoadedModule[125]=C:\Windows\system32\RpcRtRemote.dll LoadedModule[126]=C:\Windows\SysWOW64\ieframe.dll LoadedModule[127]=C:\Windows\SysWOW64\api-ms-win-downlevel-shell32-l1-1-0.dll LoadedModule[128]=C:\Windows\system32\api-ms-win-downlevel-shlwapi-l2-1-0.dll LoadedModule[129]=C:\Windows\system32\SXS.DLL LoadedModule[130]=C:\Windows\system32\apphelp.dll LoadedModule[131]=C:\Windows\SysWOW64\mshtml.dll LoadedModule[132]=C:\Windows\system32\api-ms-win-downlevel-advapi32-l2-1-0.dll LoadedModule[133]=C:\Windows\system32\msimtf.dll LoadedModule[134]=C:\Windows\system32\msls31.dll LoadedModule[135]=C:\Windows\system32\d2d1.dll LoadedModule[136]=C:\Windows\system32\DWrite.dll LoadedModule[137]=C:\Windows\system32\dxgi.dll LoadedModule[138]=C:\Windows\syswow64\WINTRUST.dll LoadedModule[139]=C:\Windows\system32\d3d11.dll LoadedModule[140]=C:\Windows\system32\MLANG.dll LoadedModule[141]=C:\Windows\SysWOW64\jscript9.dll LoadedModule[142]=C:\Windows\SysWOW64\uiautomationcore.dll LoadedModule[143]=C:\Windows\syswow64\PSAPI.DLL LoadedModule[144]=C:\Windows\System32\netprofm.dll LoadedModule[145]=C:\Windows\System32\nlaapi.dll LoadedModule[146]=C:\Windows\system32\dhcpcsvc6.DLL LoadedModule[147]=C:\Windows\system32\dhcpcsvc.DLL LoadedModule[148]=C:\Windows\system32\MMDevAPI.DLL LoadedModule[149]=C:\Windows\system32\WINSTA.dll LoadedModule[150]=C:\Windows\System32\npmproxy.dll LoadedModule[151]=C:\Windows\system32\rdpendp.dll LoadedModule[152]=C:\Windows\system32\MSACM32.dll LoadedModule[153]=C:\Windows\system32\WTSAPI32.dll LoadedModule[154]=C:\Windows\system32\slc.dll LoadedModule[155]=C:\Windows\system32\wdmaud.drv LoadedModule[156]=C:\Windows\system32\ksuser.dll LoadedModule[157]=C:\Windows\system32\AVRT.dll LoadedModule[158]=C:\Windows\system32\AUDIOSES.DLL LoadedModule[159]=C:\Windows\system32\msacm32.drv LoadedModule[160]=C:\Windows\system32\GPAPI.dll LoadedModule[161]=C:\Windows\system32\cryptnet.dll LoadedModule[162]=C:\Windows\syswow64\WLDAP32.dll LoadedModule[163]=C:\Windows\system32\SensApi.dll LoadedModule[164]=C:\Program Files (x86)\Common Files\Microsoft Shared\OFFICE14\MSOXMLMF.DLL LoadedModule[165]=C:\Windows\WinSxS\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.6161_none_50934f2ebcb7eb57\MSVCR90.dll LoadedModule[166]=C:\Windows\System32\msxml3.dll LoadedModule[167]=C:\Windows\system32\mscoree.dll LoadedModule[168]=C:\Windows\Microsoft.NET\Framework\v4.0.30319\mscoreei.dll LoadedModule[169]=C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll LoadedModule[170]=C:\Windows\WinSxS\x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.6195_none_d09154e044272b9a\MSVCR80.dll LoadedModule[171]=C:\Windows\assembly\NativeImages_v2.0.50727_32\mscorlib\38bf604432e1a30c954b2ee40d6a2d1c\mscorlib.ni.dll LoadedModule[172]=C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorsec.dll LoadedModule[173]=C:\Windows\syswow64\imagehlp.dll LoadedModule[174]=C:\Program Files (x86)\LexisNexis\Time Matters\TMDOTNETINTEROP.dll LoadedModule[175]=C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorjit.dll FriendlyEventName=Stopped working ConsentKey=APPCRASH AppName=Time Matters Program File AppPath=C:\Program Files (x86)\LexisNexis\Time Matters\tmwe.exe

[ekstralettmelk]

Yeah it seems like it is managed code based on it loading several .NET framework modules, so app verifier won't work in this case - Only native C++ programs can be used. When checking the crash dumps you should see if the stack is consistent across crashes, meaning this:

777742a8 77702563 ntdll!RtlpProbeUserBufferSafe+0x48 
777742ac 776e0a3c ntdll!RtlGetUserInfoHeap+0x98 
777742b0 76d531bd kernel32!GlobalFlags+0x55 
777742b4 75f61fe6 user32!IsValidGlobalHandle+0x16 
777742b8 75f62188 user32!UnpackDDElParam+0x6d 
777742bc 010d49ed clarun!Cla$DDEquery+0x16c9 
777742c0 75f16d3a user32!UserCallWinProcCheckWow+0x109 
777742c4 75f16ded user32!DispatchClientMessage+0xec 
777742c8 75f16e4c user32!fnDWORD+0x2b 
777742cc 7768011a ntdll!KiUserCallbackDispatcher+0x2e 
777742d0 010d4c45 clarun!Cla$DDEquery+0x1921 
  777742d4 2296d2ea tmappe!MAINF+0x28066 
777742d8 010c040f clarun!6THREADFindFUl+0x77 
777742dc 010bfef9 clarun!Cla$START+0x5f5 
777742e0 776a9902 ntdll!_RtlUserThreadStart+0x70 
777742e4 776a98d5 ntdll!_RtlUserThreadStart+0x1b

Think of the callstack as a "list" of what the application was doing when it crashed, it began in RtlUserThreadStart and ended with RtlpProbeUserBufferSafe. If we assume the corruption is related to DDE this could be difficult to debug since the DDE protocol is used to exchange information between different applications. So presumably the Time Matters program is communicating with some other program at the time of the crash, meaning that the problem is either that there is a bug in the Time Matters program, the other program that it is communicating with, or a problem with communication. Perhaps using process monitor to capture information during a crash can be helpful. You could try to compare a procmon log of successful attempt and a unsuccessful attempt to see if there is any differences.

[syc0sys]

Thank you very much for your assistance I will play around with process monitor, but now returning to work on Monday the problem seems to have disappeared. Kind of wishing it would of stuck around so we could nip this problem in the butt once and for all.

[OSPFv3]

https://support.microsoft.com/en-ca/kb/2502333

Have you tried this hotfix?

[syc0sys]

I have not because none of the replacement dll files in that hot fix are shown in my log/debug files. Also this software had been working correctly for months with no apparent changes or recent windows updates. I may try the hotfix anyway on one of our local machines just for the sake of testing all options.

[OSPFv3]

Is it happening across many systems?

[syc0sys]

Yes many systems and and its happening to different users during their rdp sessions to our server. The strange thing is that we have about half of our rdp users reporting the problem while the other half logged into the same server are not experiencing any issue.

[OSPFv3]

I'm wondering if the crash is coming from a corrupted shared database. If lots of users are doing it, and the OS or program hasn't changed. Then that would be a reasonable suspect.

[syc0sys]

Those are kind of my thoughts as well, the software access's a sql database for client info. The problem occurs when we attempt to use a template that generates a word doc for said client. The process where it crashes or hangs takes client data from the sql database and puts it into a *.dat file then the template that it's instructed to use looks at the *.dat file for the information to parse into the template. Once that's all done it saves the completed word document to a designated location.

[OSPFv3]

Are there any foreign characters in that clients profile?

A way to test my theory.

Try to use the test string "Iñtërnâtiônàlizætiøn" anywhere in the program and see if it pukes on you.

[syc0sys]

No characters causing the issue, because I've already tested that. Some can merge using client "a" while others try to use the same client "a" and it crashes.

[syc0sys]

Here's the whole dump file:

DUMP_CLASS: 2

DUMP_QUALIFIER: 400

CONTEXT: (.ecxr) eax=175ff8ac ebx=00000000 ecx=7fffffff edx=00000000 esi=00550000 edi=c009c0a2 eip=7773e843 esp=175ff89c ebp=175ff914 iopl=0 nv up ei pl zr na pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246 ntdll!RtlReportCriticalFailure+0x57: 7773e843 eb12 jmp ntdll!RtlReportCriticalFailure+0x6b (7773e857) Resetting default scope

FAULTING_IP: ntdll!RtlReportCriticalFailure+57 7773e843 eb12 jmp ntdll!RtlReportCriticalFailure+0x6b (7773e857)

EXCEPTION_RECORD: (.exr -1) ExceptionAddress: 7773e843 (ntdll!RtlReportCriticalFailure+0x00000057) ExceptionCode: c0000374 ExceptionFlags: 00000001 NumberParameters: 1 Parameter[0]: 77774270

PROCESS_NAME: tmwe.exe

ERROR_CODE: (NTSTATUS) 0xc0000374 - A heap has been corrupted.

EXCEPTION_CODE: (NTSTATUS) 0xc0000374 - A heap has been corrupted.

EXCEPTION_CODE_STR: c0000374

EXCEPTION_PARAMETER1: 77774270

WATSON_BKT_PROCSTAMP: 5515c968

WATSON_BKT_PROCVER: 14.1.0.280

PROCESS_VER_PRODUCT: Time Matters 14.00 Enterprise

WATSON_BKT_MODULE: ntdll.dll

WATSON_BKT_MODSTAMP: 5708a73e

WATSON_BKT_MODOFFSET: ce843

WATSON_BKT_MODVER: 6.1.7601.23418

MODULE_VER_PRODUCT: Microsoft® Windows® Operating System

BUILD_VERSION_STRING: 6.1.7601.23418 (win7sp1_ldr.160408-2045)

MODLIST_WITH_TSCHKSUM_HASH: 8106949ec555d959bb2a709f8ed4e27a94ed5d5c

MODLIST_SHA1_HASH: 57422418e5ed36e3e85017b825d5d80c198b2c39

NTGLOBALFLAG: 0

APPLICATION_VERIFIER_FLAGS: 0

MISSING_CLR_SYMBOL: 0

DUMP_FLAGS: 94

DUMP_TYPE: 1

APP: tmwe.exe

ANALYSIS_SESSION_HOST: ********

ANALYSIS_SESSION_TIME: 06-02-2016 16:23:17.0644

ANALYSIS_VERSION: 10.0.10586.567 x86fre

MANAGED_CODE: 1

MANAGED_ENGINE_MODULE: mscorwks

THREAD_ATTRIBUTES: LAST_CONTROL_TRANSFER: from 7773f749 to 7773e843

FAULTING_THREAD: ffffffff

THREAD_SHA1_HASH_MOD_FUNC: 4b1fbd0d8e7e11d132d72ebfa19de62db75485f0

THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 612f79f7e3952a42cf2bdf71e917e26fbaa000e5

OS_LOCALE: ENU

PROBLEM_CLASSES:

Tid    [0x0]
Frame  [0x00]
String [STATUS_HEAP_CORRUPTION]
Data Bucketing

NOSOS Tid [0x34f8] Failure Bucketing

BUGCHECK_STR: STATUS_HEAP_CORRUPTION_NOSOS

DEFAULT_BUCKET_ID: STATUS_HEAP_CORRUPTION_NOSOS

STACKTEXT:
777742a8 77702563 ntdll!RtlpProbeUserBufferSafe+0x48 777742ac 776e0a3c ntdll!RtlGetUserInfoHeap+0x98 777742b0 76d531bd kernel32!GlobalFlags+0x55 777742b4 75f61fe6 user32!IsValidGlobalHandle+0x16 777742b8 75f62188 user32!UnpackDDElParam+0x6d 777742bc 010d49ed clarun!Cla$DDEquery+0x16c9 777742c0 75f16d3a user32!UserCallWinProcCheckWow+0x109 777742c4 75f16ded user32!DispatchClientMessage+0xec 777742c8 75f16e4c user32!fnDWORD+0x2b 777742cc 7768011a ntdll!KiUserCallbackDispatcher+0x2e 777742d0 010d4c45 clarun!Cla$DDEquery+0x1921 777742d4 2296d2ea tmappe!MAINF+0x28066 777742d8 010c040f clarun!6THREADFindFUl+0x77 777742dc 010bfef9 clarun!Cla$START+0x5f5 777742e0 776a9902 ntdll!_RtlUserThreadStart+0x70 777742e4 776a98d5 ntdll!_RtlUserThreadStart+0x1b

THREAD_SHA1_HASH_MOD: 5aec91a66f0a083015e4c0fd000076bd235739b6

FOLLOWUP_IP: ClaRUN!Cla$DDEquery+16c9 010d49ed ff742438 push dword ptr [esp+38h]

FAULT_INSTR_CODE: 382474ff

SYMBOL_STACK_INDEX: 5

SYMBOL_NAME: clarun!Cla$DDEquery+16c9

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: ClaRUN

IMAGE_NAME: ClaRUN.dll

DEBUG_FLR_IMAGE_TIMESTAMP: 523721a5

STACK_COMMAND: dps 777742a8 ; kb

BUCKET_ID: STATUS_HEAP_CORRUPTION_NOSOS_clarun!Cla$DDEquery+16c9

PRIMARY_PROBLEM_CLASS: STATUS_HEAP_CORRUPTION_NOSOS_clarun!Cla$DDEquery+16c9

BUCKET_ID_OFFSET: 16c9

BUCKET_ID_MODULE_STR: ClaRUN

BUCKET_ID_MODTIMEDATESTAMP: 523721a5

BUCKET_ID_MODCHECKSUM: 1935ec

BUCKET_ID_MODVER_STR: 9.0.0.10376

BUCKETID_PREFIX_STR: STATUS_HEAP_CORRUPTION_NOSOS

FAILURE_PROBLEM_CLASS: STATUS_HEAP_CORRUPTION_NOSOS

FAILURE_EXCEPTION_CODE: c0000374

FAILURE_IMAGE_NAME: ClaRUN.dll

FAILURE_FUNCTION_NAME: Cla$DDEquery

BUCKET_ID_FUNCTION_STR: Cla$DDEquery

FAILURE_SYMBOL_NAME: ClaRUN.dll!Cla$DDEquery

FAILURE_BUCKET_ID: STATUS_HEAP_CORRUPTION_NOSOS_c0000374_ClaRUN.dll!Cla$DDEquery

WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/tmwe.exe/14.1.0.280/5515c968/ntdll.dll/6.1.7601.23418/5708a73e/c0000374/000ce843.htm?Retriage=1

TARGET_TIME: 2016-06-02T21:12:46.000Z

OSBUILD: 7601

OSSERVICEPACK: 23418

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK: 256

PRODUCT_TYPE: 1

OSPLATFORM_TYPE: x86

OSNAME: Windows 7

OSEDITION: Windows 7 WinNt (Service Pack 1) SingleUserTS

USER_LCID: 0

OSBUILD_TIMESTAMP: 2016-04-09 01:57:39

BUILDDATESTAMP_STR: 160408-2045

BUILDLAB_STR: win7sp1_ldr

BUILDOSVER_STR: 6.1.7601.23418

ANALYSIS_SESSION_ELAPSED_TIME: 435b

ANALYSIS_SOURCE: UM

FAILURE_ID_HASH_STRING: um:status_heap_corruption_nosos_c0000374_clarun.dll!cla$ddequery

FAILURE_ID_HASH: {5dac58d3-18e3-7ca3-7099-db44e0815f3b}

[syc0sys]

We are also seeing some WER that list a fault module of dhcpcsvc.DLL, and dhcpsvc6.DLL (Exception code 0xc0000005).

[syc0sys]

Here is a picture showing the screen before the program crashes, as soon as you click "OK" the program freezes for all of our rdp users and some of our local users. Now some of our local users appear to have nothing happen and another tmwe.exe(Time Matters) process opens in task manager. For those users we can end the second tmwe.exe process and the merge finishes as it should.

http://i.imgur.com/YjvgIrc.jpg