Go Back   Cockos Incorporated Forums > REAPER Forums > REAPER Compatibility

Reply
 
Thread Tools Display Modes
Old 01-14-2009, 12:42 PM   #1
the all new rob
Human being with feelings
 
the all new rob's Avatar
 
Join Date: Dec 2007
Location: east coast of Kansas
Posts: 681
Default Using dpclat and RATT to diagnose audio performance problems

This is intended to be a friendly guide to using these two fine (and free!) tools to diagnose (not fix) problems with clicking and popping audio.

dpclat is a tool for visualizing the latency of deferred procedure calls by kernel-mode device drivers. In doing so, it also makes a good assessment tool for your system's ability to handle streams of audio or video. It's extremely well described on the linked page, so please do read that page and familiarize yourself with the background.

To use dpclat, you just run the executable (there is no installation needed). You will get a window with an updating bar graph showing you the time taken for deferred procedure calls (DPC) to all the drivers in the system. What it does not show is which driver is responsible if you have a problem.

The tool will give you a sort of "go/no go" visualization. If you get all green bars, your system (hardware and installed drivers) is generally well-equipped for streaming. If you get red bars, the tool is alerting you to the presence of a problem, but that's about it--finding which driver is responsible will be done with another tool altogether.

RATTV3 is that other tool. As described on the linked page, "RATTV3 is designed to help developers of drivers and other kernel mode components audit the ISR and DPC execution time of their components."

However, we will use it a little differently. In cases where we have red bars showing in dpclat window, we'll use RATT to find out where to start looking for the problem driver(s). What we do once we ind them is out of scope for this tutorial--I'm just showing you how to use the tool.

RATT gets installed and then runs as a system tray icon. It will start every time you boot until uninstalled, so make sure you are paying attention and don't leave it running after you are done troubleshooting.

With RATT running, take whatever steps are needed to reproduce your audio issue. You'll want to make sure to leave RATT on for at least three minutes to ensure you get a human-readable log file built. RATT will build a log file in the directory %system32%\logfiles\rattv3 based on the info it collects about DPC times. (note: %system32% is an environment variable--it can point to a different physical path depending on how you chose to install Windows. Look for a folder system32 under the folder where windows is installed.)

You then will open up the above directory in explorer, and open a text file named <machine-name>.cswa-accumulator-report.txt. Wordpad works much better than notepad for reading this file.

Inside this file you'll find a section for each kernel mode driver (DRIVERNAME.SYS). Each section is a histogram (sort of) of DPC times. There are 1109 possible "buckets" a driver call can fall into. A bucket is a range of duration times. What we are looking for is when calls to a given driver take longer than a few hundred microseconds. when we find that, we have identified a driver that could be behaving better.

The easiest way to spot the misbehavior is to look at the first column. the buckets we don't want to see will be numbered above 100. Ignore anything with a bucket number under 50. Those are too fast to matter. Any driver with items in buckets numbered over 109 is doing DPCs that take longer than a millisecond (1000 microseconds), and may be contributing to our performance issues. Further troubleshooting will be needed to determine that.

So a "healthy" driver would have a histogram like this:
Code:
Cumulative Histogram for:      Ntfs.sys  (Ntfs.sys)
  #,                  Range, Label,       ISR Count,  DPC Count,  DPCTmr Count
 10,   10.00us to   19.99us, <20.00us,            0,          0,            63
 11,   20.00us to   29.99us, <30.00us,            0,          0,            11
                              TOTALS,             0,          0,            74
And a driver that deserves further scrutiny would appear like this:
Code:
Cumulative Histogram for:     atapi.sys  (atapi.sys)
  #,                  Range, Label,       ISR Count,  DPC Count,  DPCTmr Count
 11,   20.00us to   29.99us, <30.00us,            0,       2269,             0
 12,   30.00us to   39.99us, <40.00us,            0,       4576,             0
 13,   40.00us to   49.99us, <50.00us,            0,        446,             0
 14,   50.00us to   59.99us, <60.00us,            0,         70,             0
 15,   60.00us to   69.99us, <70.00us,            0,         82,             0
 16,   70.00us to   79.99us, <80.00us,            0,         49,             0
 17,   80.00us to   89.99us, <90.00us,            0,         24,             0
 18,   90.00us to   99.99us, <100.00us,           0,         41,             0
 19,  100.00us to  109.99us, <110.00us,           0,        136,             0
[more like this...]
302,    2.93ms to    2.94ms, <2.94ms,             1,          0,             0
342,    3.33ms to    3.34ms, <3.34ms,             1,          0,             0
351,    3.42ms to    3.43ms, <3.43ms,             1,          0,             0
415,    4.06ms to    4.07ms, <4.07ms,             1,          0,             0
                              TOTALS,          7841,       7841,             0
So, low bucket numbers: good, high bucket numbers: bad.

There are also three different times being analyzed by RATT: execution times of interrupt service routines (ISR), deferred procedure calls (DPC), and timer DPCs (DPCTmr). They are different, but effectively all are covered by the low bucket numbers: good, high bucket numbers: bad mantra above.


What next, once you've drawn a bead on a specific driver?
+ Selectively disable suspect devices with device manager to see if your issues get cleared up.
+ get updated drivers for the component if possible.
+ check the hardware behind the driver for performance issues.

It's worth saying--long times aren't a smoking gun here, they are a clue to the real issue. They should be used to narrow down the search field for your cause of your performance issue--it's still up to you to determine the resolution..

Hopefully this is of use to some of you.
the all new rob is offline   Reply With Quote
Old 01-14-2009, 03:00 PM   #2
Ollie
Super Moderator (no feelings)
 
Ollie's Avatar
 
Join Date: Dec 2007
Location: On or near a dike
Posts: 9,739
Default

Good job! I bookmarked this in my "Reaper resources" folder and linked to it for some poor guys who have DPC troubles with their new Lenovo Thinkpad SL laptops.
Ollie is offline   Reply With Quote
Old 01-14-2009, 04:44 PM   #3
dub3000
Human being with feelings
 
dub3000's Avatar
 
Join Date: Mar 2008
Location: Sydney, Australia
Posts: 3,831
Default

great post. can this be made sticky?
dub3000 is offline   Reply With Quote
Old 01-16-2009, 05:08 PM   #4
politcat
Human being with feelings
 
politcat's Avatar
 
Join Date: Jul 2006
Location: stuck in transition
Posts: 1,850
Default

yikes! I ran ratt and I have several drivers above 109. but i'm stuck now cuz I don't know how to disable any of them and i'm tired of googling for an answer. ???

HDAudBus.sys
portcls.sys
rdbss.sys
sthda.sys
tcpip.sys
TDI.SYS
USBPORT.SYS
ntkrnlpa.exe
ACPI.sys
atapi.sys
NDIS.sys

Last edited by politcat; 01-16-2009 at 05:25 PM.
politcat is offline   Reply With Quote
Old 01-16-2009, 10:44 PM   #5
the all new rob
Human being with feelings
 
the all new rob's Avatar
 
Join Date: Dec 2007
Location: east coast of Kansas
Posts: 681
Default

Out of your list, the thing that kind of jumps out at me is the first one. That's the stock driver for the Intel chipset onboard audio for your motherboard.

If you are using the onboard audio chip, it may not be feasible (as a solution) to disable it, but I'd try deactivating your onboard sound in the BIOS setup screen (as a diagnostic).
__________________
"Well feeling (emotion) combined with an artist's discipline is the rarest thing in the world."
-- Ursula Nordstrom
the all new rob is offline   Reply With Quote
Old 01-17-2009, 10:10 AM   #6
politcat
Human being with feelings
 
politcat's Avatar
 
Join Date: Jul 2006
Location: stuck in transition
Posts: 1,850
Default

I didn't see anywhere in the bios setup screen that the onboard sound can be disabled
politcat is offline   Reply With Quote
Old 02-03-2009, 04:47 AM   #7
SliderJeff
Human being with feelings
 
Join Date: Jun 2008
Posts: 9
Default

So,

Does anyone have any thoughts on tracking down a driver that's listed as Unknown which is causing the issue? Here's the details:

Cumulative Histogram for: Unknown (Unknown)

# Range Label ISR Count DPC Count DPCTmr Count
0 0.00us to 0.99us <1.00us 0 1882 0
1 1.00us to 1.99us <2.00us 7015368 37918 2
2 2.00us to 2.99us <3.00us 32596420 164608 434
3 3.00us to 3.99us <4.00us 1064039 157747 50773
4 4.00us to 4.99us <5.00us 46677 72477 108547
5 5.00us to 5.99us <6.00us 137372 133734 32974
6 6.00us to 6.99us <7.00us 134460 64185 98156
7 7.00us to 7.99us <8.00us 226479 43175 40970
8 8.00us to 8.99us <9.00us 147192 16215 51761
9 9.00us to 9.99us <10.00us 80008 10062 17049
10 10.00us to 19.99us <20.00us 527600 51398 134029
11 20.00us to 29.99us <30.00us 375807 234711 105331
12 30.00us to 39.99us <40.00us 14491 104751 12448
13 40.00us to 49.99us <50.00us 212 61401 203
14 50.00us to 59.99us <60.00us 1 110689 9
15 60.00us to 69.99us <70.00us 0 128944 0
16 70.00us to 79.99us <80.00us 0 10250 0
17 80.00us to 89.99us <90.00us 0 2157 0
18 90.00us to 99.99us <100.00us 0 631 0
19 100.00us to 109.99us <110.00us 0 337 0
20 110.00us to 119.99us <120.00us 0 117 0
21 120.00us to 129.99us <130.00us 14 13 0
22 130.00us to 139.99us <140.00us 8 7 0
23 140.00us to 149.99us <150.00us 2 9 0
24 150.00us to 159.99us <160.00us 0 1 0
25 160.00us to 169.99us <170.00us 0 2 0
26 170.00us to 179.99us <180.00us 0 1 0
28 190.00us to 199.99us <200.00us 0 1 0
29 200.00us to 209.99us <210.00us 0 1 0
37 280.00us to 289.99us <290.00us 0 1 0
1011 30.00ms to 39.99ms <40.00ms 0 1 0
1012 40.00ms to 49.99ms <50.00ms 0 10 0
1013 50.00ms to 59.99ms <60.00ms 0 2 0
1019 110.00ms to 119.99ms <120.00ms 1 0 0
1020 120.00ms to 129.99ms <130.00ms 1 0 0
1021 130.00ms to 139.99ms <140.00ms 0 8 0
1022 140.00ms to 149.99ms <150.00ms 0 2 0
1027 190.00ms to 199.99ms <200.00ms 0 2 0
TOTALS 42366152 1407450 652686

According to the Rattv3 docs, the "Unknown" thing is a known bug, but I doubt it will ever be fixed.

Thanks,
Jeff
SliderJeff is offline   Reply With Quote
Old 03-17-2009, 10:10 PM   #8
pipelineaudio
Mortal
 
pipelineaudio's Avatar
 
Join Date: Jan 2006
Location: Wickenburg, Arizona
Posts: 13,444
Default

Please come to chat sometime
__________________
REAPER Shirts are Back! - https://shop.spreadshirt.com/pipelineaudio/
pipelineaudio is offline   Reply With Quote
Old 03-18-2009, 04:35 PM   #9
pipelineaudio
Mortal
 
pipelineaudio's Avatar
 
Join Date: Jan 2006
Location: Wickenburg, Arizona
Posts: 13,444
Default

can someone help me with mine?

https://stash.reaper.fm/oldsb/131691/ratt-report.zip

so far it looks like

ipnat.sys
videoprt.sys
portcls.sys
sthda.sys (no surprise there)
tcpip.sys
usbprt.sys (youch, I often use a usb soundcard!)
ntkrnlpa.sys
acpi.sys
atapi.sys
iaStor.sys
NDIS.sys

at the very least are the ones messing with me...what can I do?
__________________
REAPER Shirts are Back! - https://shop.spreadshirt.com/pipelineaudio/
pipelineaudio is offline   Reply With Quote
Old 03-18-2009, 07:07 PM   #10
the all new rob
Human being with feelings
 
the all new rob's Avatar
 
Join Date: Dec 2007
Location: east coast of Kansas
Posts: 681
Default

I'd go into device manager and disable your SigmaTel onboard audio. That latency is ridiculous. And you seem to be using a MOTU card for the "Real" audio. It may be necessary to hit the BIOS to turn it all the way off.

I'd also take a look at disabling the nic. The NDIS.sys numbers are insane.

Do you have any USB stuff attached? I would look for any high-resolution mice or things like that and pull them if the above don't help, but I think those two are the likeliest sources of problems.
__________________
"Well feeling (emotion) combined with an artist's discipline is the rarest thing in the world."
-- Ursula Nordstrom
the all new rob is offline   Reply With Quote
Old 03-19-2009, 03:16 PM   #11
pipelineaudio
Mortal
 
pipelineaudio's Avatar
 
Join Date: Jan 2006
Location: Wickenburg, Arizona
Posts: 13,444
Default

whats nic and ndis?

I use the HDA audio when I am not using real soundcards...how did you know about the motu? Are motu drivers running even though the motu interface is 90 miles away? the run had no usb devices plugged in while rat was monitoring
__________________
REAPER Shirts are Back! - https://shop.spreadshirt.com/pipelineaudio/
pipelineaudio is offline   Reply With Quote
Old 05-30-2010, 06:50 AM   #12
steveecrane
Human being with feelings
 
steveecrane's Avatar
 
Join Date: May 2010
Location: Staffordshire, England, UK
Posts: 1
Default DPCs, ISRs, Latency, DPCLat and RATTv3

Guys,

This has proven useful:

http://www.resplendence.com/latmon

in tracking down problematic services, et al. You may want to take a look!

I've found it more useful (and more 'accurate'!?) than "DPCLat.exe" by TheSyCon in identifying dodgy routines/drivers and less-intensive (in terms of its 'learning curve') than "RATT".

There are 3 Tabs: a "Latency" overview, an "Activity" tab (the useful one, with all the ISR/DPC detail) and one with some perfunctory info about "Drivers".

From a purely personal perspective, I have stopped all networking hardware and all related services, but have a major problem with TCPIP.sys interrupting my music with glitches through copious DPCs (as well as moderate - but presumably necessary - nonsense from ACPI.sys too) on my Acer 7720G (3GB, Vista, fully-patched).

Anyone have any ideas on why TCPIP.sys is such a problem and/or whether I can disable it entirely (probably through SysInternals' "AutoRuns" app)?

I dual-boot this Vista machine - and am lucky enough to actually have a couple of physical Acronis-managed boot drives in it (rather than multiple Windows partitions on a single physical drive) - but, if necessary, I don't mind having no Internet access on the 'Music DAW' drive, if it will cure the nasty and unpredictable audio behaviour on such a well-spec'd laptop.

Whilst I'd like to have the ability for network access from this laptop, when booted as a Music DAW - for s/w updates, maintenance, et al - I'll readily forsake it in order to make some half-decent music!!!

So, what to do ...?

All reasonable help welcomed.

Cheers,

Steve.
steveecrane is offline   Reply With Quote
Old 06-18-2010, 04:58 AM   #13
RCB90028
Human being with feelings
 
Join Date: May 2010
Posts: 17
Default

I'm running Windows Vista Premium on a Compaq Presario CQ60-224 NR laptop with 4 gigs o' ram, dual Intel T3400 2.16/2.17 mHz, onboard Intel 4500 graphics chip, and a Tascam US-144mkii USB interface.

After a factory install with up-to-date drivers, Reaper and GTR3 sound horrible. After making major changes to the settings as described in this and other threads, I've gotten a HUGE performance boost. I went from lots of red spikes in dpcl to an occassional spike, mostly green with some yellow. Currently, I'm running at anywhere between 51us, mosty around 1000, with hardly anything over 2000, with an occasional spike...one hit 3006us during the writing of this post. It looks like one driver is real bad, the rest...

I installed RATT. Here's the results:
--------------------------------------------------------------------------------
| Counts by driver for: Laptop
| From: FRI JUN-18-2010 11:18:6.69
| To: FRI JUN-18-2010 11:36:1.16
| Runs: 5
--------------------------------------------------------------------------------
Cumulative Histogram for: ()
#, Range, Label, ISR Count, DPC Count, DPCTmr Count
1, 1.00us to 1.99us, <2.00us, 40, 0, 0
2, 2.00us to 2.99us, <3.00us, 5227, 0, 0
3, 3.00us to 3.99us, <4.00us, 438650, 0, 0
4, 4.00us to 4.99us, <5.00us, 1140147, 0, 0
5, 5.00us to 5.99us, <6.00us, 909959, 0, 0
6, 6.00us to 6.99us, <7.00us, 674598, 0, 0
7, 7.00us to 7.99us, <8.00us, 253542, 0, 0
8, 8.00us to 8.99us, <9.00us, 41236, 0, 0
9, 9.00us to 9.99us, <10.00us, 22701, 0, 0
10, 10.00us to 19.99us, <20.00us, 61242, 0, 0
11, 20.00us to 29.99us, <30.00us, 5593, 0, 0
12, 30.00us to 39.99us, <40.00us, 635, 0, 0
13, 40.00us to 49.99us, <50.00us, 185, 0, 0
14, 50.00us to 59.99us, <60.00us, 31, 0, 0
15, 60.00us to 69.99us, <70.00us, 1, 0, 0
21, 120.00us to 129.99us, <130.00us, 173, 0, 0
22, 130.00us to 139.99us, <140.00us, 2519, 0, 0
23, 140.00us to 149.99us, <150.00us, 3115, 0, 0
24, 150.00us to 159.99us, <160.00us, 1072, 0, 0
25, 160.00us to 169.99us, <170.00us, 1207, 0, 0
26, 170.00us to 179.99us, <180.00us, 1132, 0, 0
27, 180.00us to 189.99us, <190.00us, 907, 0, 0
28, 190.00us to 199.99us, <200.00us, 551, 0, 0
29, 200.00us to 209.99us, <210.00us, 187, 0, 0
30, 210.00us to 219.99us, <220.00us, 44, 0, 0
31, 220.00us to 229.99us, <230.00us, 28, 0, 0
32, 230.00us to 239.99us, <240.00us, 14, 0, 0
33, 240.00us to 249.99us, <250.00us, 6, 0, 0
34, 250.00us to 259.99us, <260.00us, 13, 0, 0
35, 260.00us to 269.99us, <270.00us, 5, 0, 0
36, 270.00us to 279.99us, <280.00us, 1, 0, 0
37, 280.00us to 289.99us, <290.00us, 3, 0, 0
38, 290.00us to 299.99us, <300.00us, 2, 0, 0
39, 300.00us to 309.99us, <310.00us, 1, 0, 0
40, 310.00us to 319.99us, <320.00us, 1, 0, 0
42, 330.00us to 339.99us, <340.00us, 3, 0, 0
44, 350.00us to 359.99us, <360.00us, 1, 0, 0
48, 390.00us to 399.99us, <400.00us, 1, 0, 0
TOTALS, 3564773, 0, 0

I've disabled the nic, cam, USB card reader, onboard fax, and the onboard sound. This helped, but not much.

What really improved performance was turning off the page file and a lower screen resolution. After doing these two things, it flies...as compared to the factory install.

Does the report show where the problem is (it's all gibberish to me)?

Thanks!
RCB90028 is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 05:24 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.