SCSI ID Precedence and the Impact on Performance


Abstract

SCSI ID precedence is defined in the SCSI specification, and while the assignment of SCSI IDs  is not defined in that specification, SCSI vendors standardized on a plan that the Initiator default to ID 7, while the boot drive take ID 0.


I've seen no record of this being a reasoned decision, however, or of the thinking behind it. Does this make sense for performance reasons? Was it ideal when SCSI's 5MB/s data transfer rate was considered fast? And if so, does it still apply at today's data rates? Or was it inspired by some long forgotten logic: start at 0 and count up - this means that the first ID found is the one to boot off of, which also simplifies adding devices at a later point - it is less likely that the boot drive will be bumped from C: to D:, or /dev/sda to /dev/sdb?  Or is it merely convenience in configuration? Someone did it first, and everyone followed the pattern, and now we have all come to expect it.  While internal SCSI devices can have their IDs configured via jumpers, external Jaz and Zip drives only give you the choice of 5 or 6. Why are we limited to these choices? Is it a good idea to give a slow device a higher priority (as we are required to do with Jaz and Zip drives), so that it gets the chance to put its data on the bus whenever it is ready, so that it can go back to being idle or searching on the next data block? And what about other characteristics of devices? Access time, cache size, data transfer rate all affect the "speed". What preference should we give the initiator and a primary data drive, quite possibly the fastest on the bus for data transfer rates, and arguably, the devices with the greatest need for speed? Do users even think about how they assign IDs?


These are questions that affect the effective planning of an efficient storage subsystem, and I haven't see a published or recognized study to address these questions.

Introduction

Parallel SCSI is both a transportation and a communication protocol designed for standardized data transfer between a computer system and a variety of peripherals (including hard disk drives, cd-roms, dvd-roms, printers, scanners, tape drives, etc) based on a client/server model. Devices are referred to as either initiators (host bus adapters) receiving commands from the OS via a driver, or targets of those commands (storage devices, printers, etc). Because multiple devices can be on the same bus, there must be some  method to determine which device will be able to use the bus at any given time. This method is arbitration based on ID precedence. When the SCSI bus is in a Bus Free state for 800 ns or longer, any device wishing to control the bus asserts its own ID. The winner of arbitration is recognized according to the priorities specified in the SCSI specification. Because adapter and peripheral manufacturers don't know what other devices you will have on your system, users of the SCSI protocol rely on configurable IDs and every device on the bus is assigned a unique ID.

The original SCSI bus design was for an 8 bit wide bus. This allowed for 8 IDs (0-7) and one byte parallel data transfers. ID 7 was designated as the highest priority ID, down to ID 0, the lowest. Later enhancements to the SCSI protocol defined 16 and 32 bit wide buses (although the 32 bit wide bus actually required two cables, and was never widely adopted), to transfer two and four bytes at a time. Due to the lack of support for the 32 bit wide bus, we can simply refer to the 8 bit bus as "narrow", and the 16 bit bus as "wide". A wide SCSI bus has 16 available IDs (0-15), and hence can support 16 devices. Since both wide and narrow devices can be on the same bus, ID precedence, which was originally defined as being 7 (highest) and 0 (lowest), must still be respected, and so on a wide bus the priority is defined as  7->0 and 15->8. (Because if it were 15 -> 0, a narrow device would not be able to recognize that it had lost arbitration to a device whose ID was above 7.) In addition to being able to negotiate for width with every device, each device on the bus also negotiates for transfer rate individually. But this SCSI bus transfer rate, or burst rate, is rarely the limiting factor. We must also consider the sustained data rate of a specific device. Both the negotiated transfer rate and the sustained data rate affect utilization of a SCSI bus. Most devices support disconnects - this allows a device with slow access time (and/or slow sustained data rate) to free up the bus while reading or writing the requested data, and internally buffer data which is then transferred at the burst rate. So, the question I pose is: How does precedence impact performance? Would an initiator with highest precedence flood a drive with requests, grabbing the bus every time it is free to pump down another request, and not let the drive, with lower precedence, provide the requested data in a timely matter? How do device and load combinations affect performance? I plan to address these questions in the following implementation, doing reads from a combination of hard disk drives and a cd-rom.


Implementation

To set up this test, I used three systems. One system, labeled "Monitor" for reference on the diagram,was purely a monitoring station, to verify that conditions on the bus were remaining under the load I had set up. This system was running DOS and a program, Resetmon, which allowed me to look at the load on the bus (the percentage of time that the bus was busy). This monitoring program was non-intrusive. Data on a SCSI bus is transferred by electrical signals transmitted in both directions; at each end of the bus is a terminator, to control reflection and preserve signal integrity. All devices read the data on the bus as it goes by to determine if another device wants to communicate with them, or to determine the state of the bus if they need to arbitrate for control. Thus, the monitoring system is able to read the SCSI signals without disturbing the other devices on the bus; since it never requests bus control or writes to the bus, the monitor is effectively invisible to the other SCSI devices. The monitoring station runs in DOS, communicating with a SCSI adapter to display statistics regarding the status of the bus to the user. I used a separate system for this monitor program because the only implementation I know of runs in DOS. I left the system running all the time so that if it did have any deleterious effect on the bus, at least it would remain constant.


One system, "Load", had an initiator (set to ID 5) and 3 hard disk drives using IDs 1, 2 and 4 and was there to provide load on the bus - with no load on the bus, there is no competition for arbitration, and there is essentially no change in the results (as will be demonstrated in Table 1). These IDs enabled me to always keep the load produced in a range where the tests I was running were for a device with higher or lower priority. This load on the SCSI bus was generated on a separate machine to ensure that the load on the processor and adapter wasn't affecting the results on the system performing the I/Os in order to insulate the results from any interference. This system was running Redhat Linux 8.0 in text mode. The drives on this system were all capable of transferring data on the SCSI bus at Ultra2 speeds - 80 MB/s, but they were set in the BIOS to not negotiate higher than 40 MB/s. This was done to ensure that the load configuration would be the same whether I was using hard disk drives (HDD) or the cd-rom (the cd-rom was only Single Ended (SE), and so the whole bus is SE when the cd-rom is on the bus; on an SE bus, no device can negotiate higher than 40 MB/s). Running the dd command [1] on two of these drives simultaneously generated a 90% load (the SCSI bus was "busy" 90% of the time). Running dd on the third drive at the same time pushed the load up to 99%.

Additionally, in order to insulate the results, all I/Os were generated using a command in the form 'dd if=/dev/sda of=/dev/null'. This command makes a disk image of the data on the drive (/dev/sda refers to the entire first SCSI drive), and sends the data to /dev/null[2] which is a special device file that swallows all data provided it. Using this method for generating I/Os allows me to make read requests to the drives and receive the data as fast as they supported, without having to worry about interference outside of the storage system as I would if I had to do anything with the data or wait on another device (compare to the case if I were reading from a drive and processing data or writing to another device in a copy). Because I needed to run 2 or 3 of these at the same time in order to generate this load, I discovered when I was setting this test up that I needed to run the command in a script that would background the process. When I tried to enter the commands manually, enough time elapsed between command issue that one process would hog the processor, and the other processes would receive less attention until it finished. By backgrounding the commands, they all started within a second of each other, and received much more evenly allocated resources.


On the third system, "Target", there is one initiator and one or two hard disk drives and/or a cd-rom. This is the system used to obtain the results that will be presented below, testing performance of the devices at different SCSI IDs. All benchmarks on this system were run using a script with the commands: "date > start_time; dd if=/dev/sda of=/dev/null; date > end_time". Subtracting the times in these files gave me elapsed time of the I/O command with in two seconds accuracy. Considering the shortest elapsed time of any test on the system was 255 seconds, this leaves me with a maximum error of less than 1%, which I believe to be accurate enough for reasonable results. Also, in order to better rely on the results, each test was run three times. In greater than 50% of the configurations, I obtained identical results on each run. On no configuration was the maximum time difference more than 1% of the mean; when the runs were different, the results in the tables below are the average of the three runs.

The SCSI bus adapter and HDDs on the Target system were capable of 160 MB/s, and this was the negotiated data rate (or the burst rate, to use the vocabulary of drive specifications) of the drives when they were alone on the bus and allowed to operate at full speed; the capacity of the HDD used was 9185 MB[3] . The negotiated data rate of the CD-ROM was 40 MB/s, and the formatted capacity of the CD used was 648 MB.


Recognizing that a benchmark is only useful in a limited sense for interpreting a specific calculation, I have attempted to limit the use of the benchmark to this goal and not to imply or assume any results that aren't valid. In order to interpret these results, I kept a consistent configuration throughout testing (other than the specific changes that made up the test cases). I changed only one thing at a time, and my results lead me to believe that it was the I/O subsystem that was the bottle neck.


Configuration


Results


Table 1 - I/O performance at high and low IDs with no load

HDD
Initiator ID
Target ID
Time required to read entire disk
Transfer rate
7
0
398 Seconds
23.08 MB/sec [4]
0
7
398 Seconds
23.08 MB/sec
CDROM
Initiator ID
Target ID
Time required to read entire disk
Transfer rate
7
0
255 Seconds
2.54 MB/sec [5]
0
7
255 Seconds
2.54 MB/sec


Table 1 shows that when there is only one device and one initiator on a bus which is more than adequate to carry the throughput, it doesn't matter which ID is higher; it is the device which is the limiting factor, not the SCSI bus. This test allows us to establish a control and determine the maximum transfer rate over time.



Table 2 - I/O performance at high and low IDs with bus at 90% load (prior to test I/Os)

HDD
Initiator ID
Target ID
Time required to read entire disk
Transfer rate
7
0
526 Seconds
17.46 MB/sec
0
7
489 Seconds
18.78 MB/sec
CDROM
Initiator ID
Target ID
Time required to read entire disk
Transfer rate
7
0
255 Seconds
2.54 MB/sec
0
7
530 Seconds
1.22 MB/sec


Table 2's data is a little more complicated. When we look at the data for the HDD, we see that with the bus under 90% load (by initiator at ID 5 and drives at IDs 1 & 2), with the initiator receiving highest precedence and the target lowest, total throughput drops by 5.5 MB/s, or nearly 25%. With the IDs reversed the throughput drops nearly as much, by 4.3 MB/s, or 19%.

Using the cd-rom, however, we see opposite results! With the initiator at highest precedence, the cd-rom still produces optimal performance, but it takes twice as long to copy the data from the device when the cd-rom has the highest precedence.

Remember that with the IDs of the devices on the Load system between 0 and 7, ID 7 will always win arbitration even if there is competition for the bus, and ID 0 will always lose arbitration anytime there is competition. These results show, as I interpret it, that for a fast device, such as this hard drive, results are slightly better when the drive as the load reliever can win the bus to return the requested data, but for a slow device, such as the cd-rom, the issue is not a matter of it winning the bus to relieve the load, but that if the initiator can't get it enough instructions to fulfill, it sits idle, while waiting for another request. The performance hit is because the initiator cannot win the bus often enough to supply the drive with enough commands to keep it busy.


Table 3 - I/O performance at high and low IDs with bus at 99% load (prior to test I/Os)

HDD
Initiator ID
Target ID
Time required to read entire disk
Transfer rate
7
0
858 Seconds
10.71 MB/sec
0
7
509 Seconds
18.05 MB/sec
CDROM
Initiator ID
Target ID
Time required to read entire disk
Transfer rate
7
0
255 Seconds
2.54 MB/sec
0
7
530 Seconds
1.22 MB/sec


As can be seen in Table 3, when the SCSI bus is under heavier load (by initiator at ID 5 and drives at IDs 1,2 & 4), performance drops further for the HDD, reflecting the increased saturation of the bus and the increased amount of time a lower ID device must wait while trying to get control of the bus, while with the cd-rom we see identical results as at the lower load in the previous test case, showing that even at this increased load, the cd-rom is able to win control of the bus often enough to provide the requested data, but the performance hit comes when the Initiator has to fight for control of the bus, and thus can't supply enough requests to the cd-rom.

For the HDD, the performance is now considerably better with the low initiator ID and high target ID than with the precedence reversed. Frankly, though, this case of a SCSI bus being utilized at 99+% (a SCSI bus will never be 100% utilized[6]) is obviously not ideal, and is a bottle neck that should be avoided anyway. It is used in this example to repeat and accentuate the results shown in Table 2.

Table 4 - I/O performance at two highest IDs with bus at 99% load (prior to test I/Os)

HDD
Initiator ID
Target ID
Time required to read entire disk
Transfer rate
7
6
400 Seconds
22.96 MB/sec
6
7
403 Seconds
22.79 MB/sec
CDROM
Initiator ID
Target ID
Time required to read entire disk
Transfer rate
7
6
255 Seconds
2.54 MB/sec
6
7
255 Seconds
2.54MB/sec

Table 4 shows the same configuration as Table 3 for the load producing system. IDs on the system under test were 6 & 7 - the two highest IDs, which reflects a different scenario - if this is the device we recognize to be under highest load and of greatest interest in terms of performance, we are enabling it to produce at top rates when receiving commands and when fulfilling them. And this is indeed what is seen. Although the HDD does show a slight drop in performance, results are close to optimal,in spite of the heavy load of the bus.


Verification of Behavior with HDD and Cd-rom

These tests showed that ID is only of significance on a heavily loaded bus (compare Table 1 to other tables), and that when optimizing IDs for the performance of a single most important device, even under 99% load, good performance can be obtained. The results appear to go against the conventional wisdom or common practice of giving the initiator the highest precedence and putting slow devices at the next highest while important devices like boot drives are at the lowest. These tests showed that the biggest influence in the transfer rate of the slow device, the cd-rom, was whether it got commands in a timely matter - the competition it faced putting its data on the bus was trivial compared to its overall transfer rate, whereas the hard disk drive was slowed under any load, but more so when it had to wait to control the bus so it could return the data, than while waiting to receive instructions. But so far I have only looked at the possibility of optimizing performance for a single device. If we combine these results simply, it would appear that optimal performance for a HDD and cd-rom would be Initiator at 7, HDD at 6, and cd-rom at 0, although HDD at 7, Initiator at 6 and cd-rom at 0 come in a close second.[7] Will it be as simple as combining individual results? In order to test configurations with both cd-rom and HDD attached on the Target system, I had to make changes to the overall setup. First, there are three possibilities for signalling on a parallel SCSI bus, Single Ended (SE), High Voltage Differential (HVD) and Low Voltage Differential (LVD). The original SCSI definition called only for SE signalling - there was one wire to transfer each bit of data, and each new generation of SCSI defined a shorter transfer period (and higher data rate) until data integrity issues over the cable length and electrical load prevented further increases (the fastest transfer rate on a SE bus is 20 MT/sec); HVD was defined to address the cable length issue, but the rise and fall time required for the transition from 0 to 5 V prohibited any increase in data rate. LVD allowed faster signal transition and higher speeds (up to 160 MT/sec , or 320 MB/sec). The HDD I've been using up to now has negotiated for 160 MB/sec transfers, on an LVD bus. When I add the cd-rom, which is an SE device, the bus reverts to SE, and so the HDD is forced to negotiate for no more than 40 MB/sec. We will see this affect performance, because where earlier the drive was transferring data at 23 MB every second, it was transfering that data in bursts of 160 MB/s, so using just under 15% of the potential for data transfers. Now that the burst transfer rate is 40 MB/s, the data transfers from this drive alone use almost 58% of the throughput available.  This (burst) transfer rate is still greater than the sustained data rate, so it doesn't cause a bottleneck when the HDD is alone on the bus, but it will affect its ability to return data when the bus is under load, so I will present all the statistics from above again for the HDD to verify the trends. Also, maximum cable length and load forced me to remove the Monitor system from the bus, so I am relying on the data I had before to know that I am applying 90% and 99% load generated by the Load system.


Table 5 - Repeat of HDD/Initiator configurations in Tables 1-4 on SE bus

No load
Initiator ID
HDD ID
Time required to read entire disk
Transfer rate
7
0
398 Seconds
23.08 MB/sec
0
7
398 Seconds
23.08 MB/sec
90% load
Initiator ID
HDD ID
Time required to read entire disk
Transfer rate
7
0
880 Seconds
10.44 MB/sec
0
7
567 Seconds 16.20 MB/sec
99% load
Initiator ID
HDD ID
Time required to read entire disk
Transfer rate
7
0
963 Seconds
9.54 MB/sec
0
7
560 seconds
16.40 MB/sec
99% load
Initiator ID
HDD ID
Time required to read entire disk
Transfer rate
7
6
533 Seconds
17.23 MB/sec
6
7
477 Seconds
19.26 MB/sec


So, in Table 5, we see for the most part that we are following the same trends established earlier: When there is no load on the bus, there is no change in data rate, either with the change in ID, or due to the change in negotiated transfer rate.
At highest and lowest IDs under 90% load, we see the same relationship between the numbers as in the first round of testing - the HDD takes longer to transfer the data when it has to fight to win control of the bus, than when the Initiator, making the data requests, has to fight for the bus.
Under 99% load, we see the same relationships, with a further drop in speed.
When using the two highest IDs for Initiator and Target, there is actually evidence of contention that I felt was statistically insignificant in the first set of tests (Table 4). The slower data transfer rate and subsequent overloading of the bus causes greater contention between the Initiator and Target - there are now different transfer rates depending on which (Initiator or Target) has the higher ID on a loaded bus; the transfer rate is slower when the Initiator has the higher ID. Even under load, one of these two devices will always win arbitration if they want to control the bus, and one would assume that these two devices with a shared goal could "work together", but it appears that the Initiator is still arbitrating for and winning the bus to send more commands down to the drive, without letting the HDD win the bus to return the requested data!
Overall, these tests show a consistent trend of better performance when the HDD has a higher precedence than the Initiator.
Now that we have established that data transfers are slower, but still following (or better defining) the general expected pattern, our hypothesis has slightly changed -  the clear advantage now seems to be HDD ID 7, Initiator ID 6 and cd-rom ID 0. To test to see if these individual top performances translate to an ideal set up, I will present all possible combinations of IDs with three devices (Initiator, HDD and cd-rom) under three conditions - no external load, 90% external load and 99% external load.

Table 6 - All possible combinations, HDD and CDROM
ID
No load
90% load
99% load

Throughput
Time required to transfer entire disk
Throughput
Time required to transfer entire disk
Throughput
Time required to transfer entire disk
Initiator 7

CDROM 0 1.23 MB/sec
528 Seconds
1.22 MB/sec
530 Seconds
1.21 MB/sec
537 Seconds
HDD 6
22.24 MB/sec
413 Seconds
17.56 MB/sec
523 Seconds
17.97 MB/sec
511 Seconds

Initiator 7

CDROM 6
2.51 MB/sec
258 Seconds
2.52 MB/sec
257 Seconds
2.55 MB/sec
254 Seconds
HDD 0
21.82 MB/sec
421 Seconds
10.33 MB/sec
889 Seconds
8.60 MB/sec
1068 Seconds

Initiator 6

CDROM 0
2.51 MB/sec
258 Seconds
1.82 MB/sec
357 Seconds
2.01 MB/sec
322 Seconds
HDD 7
21.77 MB/sec
422 Seconds
18.44 MB/sec
498 Seconds
18.48 MB/sec
497 Seconds

Initiator 6

CDROM 7
2.51 MB/sec
258 Seconds
2.51 MB/sec
258 Seconds
2.51 MB/sec
258 Seconds
HDD 0
23.08 MB/sec
398 Seconds
10.14 MB/sec
906 Seconds
8.62 MB/sec
1065 Seconds

Initiator 0

CDROM 7
2.51 MB/sec
258 Seconds
1.22 MB/sec
530 Seconds
1.22 MB/sec
530 Seconds
HDD 6
21.77 MB/sec
422 Seconds
12.92 MB/sec
711 Seconds
12.94 MB/sec
710 Seconds

Initiator 0

CDROM 6
2.52 MB/sec
257 Seconds
2.51 MB/sec
258 Seconds
2.51 MB/sec
258 Seconds
HDD 7
21.82 MB/sec
421 Seconds
14.67 MB/sec
626 Seconds
14.98 MB/sec
613 Seconds

All the results are in the table, and I have used bold to indicate the ID combinations that returned the greatest total throughput on the bus, for reads from the two devices I used in testing for each of the load conditions. These results show that on a bus under (external) load, the ideal configuration is, as proposed as a result of testing the devices individually on the bus, cd-rom at ID 0, Initiator at ID 6 and HDD at ID 7. When these devices are alone on the bus, cd-rom at ID7, Initiator at ID 6 and HDD at ID 0, produced the best results (and, in fact, performed as well as they would have if alone on the bus). Using the traditionally recommended schema for setting IDs, Initiator at ID 7, Boot drive at a low ID and "slow" device (cd-rom in this case) at a high ID didn't produce top results under any of the tested conditions.


And this got me to thinking about a couple of other important scenarios, first, that of two important devices with the same burst rate and sustained data rate. For this test, I found a second hard drive identical to the first. After rerunning the original tests to verify that this drive did perform at the same rate as the original, I tested these drives in the same ID configurations as I had with the cd-rom/HDD combination.

Table 7 - All possible combinations, two HDDs at 160MB/s
ID
No load
90% load
99% load

Throughput
Time required to transfer entire disk
Throughput
Time required to transfer entire disk
Throughput
Time required to transfer entire disk
Initiator 7

HDD 0 22.85 MB/sec
402 Seconds
10.49 MB/sec
876 Seconds
8.01 MB/sec
1146 Seconds
HDD 6
22.96 MB/sec
400 Seconds
21.26 MB/sec
432 Seconds
21.61 MB/sec
425 Seconds

Initiator 6

HDD 0
22.51 MB/sec
408 Seconds
10.25 MB/sec
896 Seconds
7.74 MB/sec
1186 Seconds
HDD 7
22.46 MB/sec
409 Seconds
18.90 MB/sec
482 Seconds
18.98 MB/sec
484 Seconds

Initiator 0

HDD 7
22.57 MB/sec
407 Seconds
10.97 MB/sec
837 Seconds
10.90 MB/sec
843 Seconds
HDD 6
22.57 MB/sec
407 Seconds
10.97 MB/sec
837 Seconds
10.90 MB/sec
843 Seconds

Table 7 shows you some results that were a bit of a surprise to me based on individual HDD tests. The best total throughput results were obtained with the Initiator set to ID 7. This was really because the HDD with the high ID was able to get such nearly optimal results that it was enough to balance out the delays suffered by the HDD at the low ID. If we felt that these drives were equally important (and we were willing to suffer the delays associated with having them on such a busy SCSI bus), we should set the Initiator ID to 0, and the two drive IDs to 6 and 7. In this configuration the performance of both suffers, but equally.

The second question that I wanted to answer, after looking at the cd-rom/HDD results, was, what really makes the cd-rom a slow device? Is it the sustained data rate, the cache size, or the burst rate? Because it behaved so much differently than the HDD under the different ID configurations, I feel that it is the cache size, or the number of commands and amount of data it can buffer while reading from the media. To test this, I wanted to set one of the two identical HDDs down to 40MB/s (still faster than the sustained data rate of the drive) to see what effect the SCSI burst rate has on throughput, when all other factors are equal.

Table 8 - All possible combinations, two HDDs at different negotiated burst rates
ID
No load
90% load
99% load

Throughput
Time required to transfer entire disk
Throughput
Time required to transfer entire disk
Throughput
Time required to transfer entire disk
Initiator 7

HDD 0 - 40 MB/s
20.01 MB/sec
459 Seconds
8.25 MB/sec
1114 Seconds
7.48 MB/sec
1228 Seconds
HDD 6 - 160 MB/s
19.63 MB/sec
468 Seconds
20.83 MB/sec
441 Seconds
20.78 MB/sec
442 Seconds

Initiator 7

HDD 6 - 40 MB/s 20.19 MB/sec
455 Seconds
18.90 MB/sec
486 Seconds
18.82 MB/sec
488 Seconds
HDD 0 - 160 MB/s 19.84 MB/sec
463 Seconds
8.94 MB/sec
1027 Seconds
7.25 MB/sec
1267 Seconds

Initiator 6

HDD 0 - 40 MB/s 19.42 MB/sec
473 Seconds
8.04 MB/sec
1143 Seconds
7.11 MB/sec
1292 Seconds
HDD 7 - 160 MB/s 19.22 MB/sec
478 Seconds
18.63 MB/sec
493 Seconds
18.71 MB/sec
491 Seconds

Initiator 6

HDD 7 - 40 MB/s 20.01 MB/sec
459 Seconds
19.67 MB/sec
467 Seconds
19.92 MB/sec
461 Seconds
HDD 0 - 160 MB/s 19.71 MB/sec
466 Seconds
9.00 MB/sec
1021 Seconds
7.32 MB/sec
1254 Seconds

Initiator 0

HDD 7 - 40 MB/s 19.97 MB/sec
460 Seconds
9.23 MB/sec
995 Seconds
9.39 MB/sec
978 Seconds
HDD 6 - 160 MB/s 19.71 MB/sec
466 Seconds
9.21 MB/sec
997 Seconds
9.37 MB/sec
980 Seconds

Initiator 0

HDD 6 - 40 MB/s 19.97 MB/sec
460 Seconds
9.26 MB/sec
992 Seconds
9.43 MB/sec
974 Seconds
HDD 7 - 160 MB/s 19.58 MB/sec
469 Seconds
9.24 MB/sec
994 Seconds
9.40 MB/sec
977 Seconds



Summary
These tests serve really to illustrate a point, rather than to mirror real world usage. I was using limited scenarios so that I could carefully track changes. And by highlighting "best total throughput" I am pretending that it is total throughput that we are tracking, and that sequential reads from cd-roms and HDD are happening simultaneously and with the same importance. When I highlight the best throughput in a row, or make statements to the effect of 'For more equal performance it would be best to set the Initiator ID to 0, and the drive IDs to 6 and 7' as I did after the 160 MB/s hard drive tests, I can only refer to this specific test. 'Optimal' will still be situation dependent, and it would probably be more realistic to set the Initiator e.g., to ID 5 (the ID just below the HDDs for which you wish to optimize). But the trends have been consistent throughout this testing, and I think they can challenge you the user to consider some alternative SCSI ID settings if you aren't getting the performance expected from your SCSI devices! (And really, if you cared about performance, how could you you let your SCSI bus become saturated, as I did in these test cases?)


[1] From the info page for dd: `dd': Convert and copy a file. `dd' copies a file (from standard input to standard output, by default) with a changeable I/O block size, while optionally performing conversions on it.


[2] From the info page for /dev/null: null, zero - data sink. Data written on a null or zero special file is discarded.


[3] Throughout this document, MB should be taken to mean 1048576 Bytes (i.e., 1024 * 1024).


[4] According to the specification sheet on this drive, sustained data rate is 18 - 25 MB/s. So, considering the range they provide and the fact that I don't know if they are talking about Mega=1000000 or 1048576, I think that it is fair to say that in this test, the drive is able to supply data at its maximum rate (i.e., the sustained data rate of the drive is the bottle neck).


[5] This cd-rom is actually marketed as 40x and Ultra, from which would, at first look, be expected to have a sustained data throughput of 6 MB/s. This drive actually uses a constant angular velocity, so 40x is the maximum speed, 17x is the minimum (according to specifications), and the average should be 27x. This would still lead to the expectation of data throughput of 4 MB/s, which was never seen from this drive. Since results were so consistent, even when the SCSI bus obviously supported higher throughput, however, 2.54 MB/s is considered to be the maximum data rate of this specific device.

[6] A properly functioning SCSI bus can't be 100% busy. This is because there is a bus phase called "bus free" (where no control or data lines are asserted) that is used to indicate that devices have the opportunity to arbitrate for control of the bus. This bus free period is a minimum of 800ns.

[7] Results in Table 4 were contrary to the previously previous two scenarios, where the best performance was achieved with the HDD at higher precedence than the Initiator; in this case we saw a minor advantage with the Initiator at the highest ID.