[weborican]

The equipment:
PowerMac G4/450
768MB RAM
ADTX K-Series RAID (http://www.adtx.com/english/k-series.html)
Adaptec Ultra2 SCSI Card
Asante GigaNIX 1000TA NIC
Netgear GS508 8-port Gigabit Switch
CAT5e cabling
Mac OS X Server 10.2.1

The problem:
Network performance is WAY lower than expected. If you read my last post on the subject (included at the end of this message) you know what the theoretical performance should be. Unfortunately those numbers don't match my results.

The Results:
- Drive to drive copy on Dual 1GHz client Machine for
reference purposes:
- Time: 29 seconds or 20.2MBps

- Finder copy of 2 files totaling 585.8MB from a 7200RPM FireWire
drive on a Dual 1GHz PowerMac:
- Server: G4/450 - 768MB RAM
Time: 37 seconds or 15.8MBps

- Server: G4/450 - 1.5GB RAM
Time: 31 seconds or 18.9MBps

- Server: G4 Dual 1-GHz - 1GB RAM (this one to a machine running OS X Client using "personal file sharing" for reference)
Time: 29 seconds or 20.2MBps

- Dual finder copies of same 2 files. Same process as above, but from 2 different Dual 1GHz PowerMacs. Some of these test were done using two NIC cards (one client connected to each one separately), but I don't recall making a noticeable impact on the copy time.
- Server: G4/450 - 768MB RAM
Time: 75 seconds - 15.6MBps

- Server: G4/450 - 1.5GB RAM
Time: 59 seconds or 19.8MBps

- Server: G4 Dual 500 - 1.5GB RAM
Time: 45 seconds or 26MBps

- Final Cut Pro Tests using the G4/450 as server and
3 simultaneous video capture from 3 different clients:
- With 1.5GB RAM and 2 NICs on server - Worked perfectly.
(2 clients connected via one NIC, third client via second one)
- With 1.5GB RAM and 1 NIC on server - Dropped frames.
- With 768RAM and 2 NICs on server - Dropped frames.
- With 768RAM and 1 NIC on server - Dropped frames.

The Conclusions/Questions:
Seems to me that CPU performance and RAM play a much bigger part on the server performance than I anticipated. I'm still 10MBps away from what I anticipated, but I believe I'll be OK as long as I outfit the server with as much RAM as it can take. Anyone know of any table or test that can show me a graph of server-performance vs. CPU speed or amount of RAM? Any other way to increase my server/network performance otherwise?

Since my "reference" copy topped-off at 20MBps, I can attribute my single-file-copy performance on the transfer-speed limitation of the FireWire IDE drives....right?

There is a 4th computer in this whole equation that is yet to be tested by capturing video at the same time as the other 3. This computer is different in that it does not have a built-in Gigabit Ethernet port, but I'm starting to believe that might not be necessary after all. My theory is that since 100Mbps should be capable of 12.5MBps this machine should be capable of keeping up with the 3.7MBps needed for DV. I tested this machine capturing video to the server by itself and everything worked fine.

Am I right in assuming that since the server is connected via Gigabit Ethernet onto a Gigabit Switch my server bottleneck is gone? If all 4 clients were connected to the switch via 100Mbps Ethernet, does the switch take care of being able to deliver 400Mbps to the server (theoretical numbers, of course)?
----
Original message:

> I have a client who need a shared storage solution for 4 Final Cut Pro
> editing systems all running under OS X. Originally they wanted to go with a
> SAN (Storage Area Network) solution, but all the currently available
> solutions are strictly OS 9 for the time being. The client is not willing to
> go back to OS 9, so I had to get creative.
>
> From what I've been able to gather, if I have an OS X Server machine running
> on a PowerMac G4/450 with an Ultra160 SCSI RAID Level 5 storage system that
> is capable of 130 MB/second (the Medea RTR 320GB drives at http://medea.com)
> I could easily achieve about 30 MB/second on a 'write' to that server. Part
> of this information comes from:
>
> http://www.xlr8yourmac.com/feedback/..._vs_snap_.html
>
> So, since DV only requires roughly 3.7 MB/second...
>
> http://www.insanely-great.com/infobank/firewirehd.html
>
> ...this means that 4 FCP stations should EASILY handle the worst case
> scenario of all 4 stations capturing video directly to the shared drive at
> the same time. Right? Am I missing anything here?

[weborican]

I have some new information I've gathered with some further testing. First, I left the G4/450 and 768MB of RAM installed with two Asante GigaNIX cards. I connected 2 of the clients via one of the NICs and the other two to the second one. The result after one day was that it worked sometimes, but had "dropped frames" problems.

I then added more RAM (1.5GB in total) to the same setup. Again, dropped frames were happening. Although this doesn't happen all of the time, it happens enough (once every two or three clips) to make the system unusable. So, although my initial tests showed I could have 3 systems writing a few minutes of video to the server at the same time, it seems like left alone long enough frames would have been dropped.

I havce now replaced the server with a G4 Dual 500 machine with 15GB of RAM and I'm crossing my fingers that it will work. There are two things that are being tested here: 1) Faster and dual processors, 2) Built-in Gigabit Ethernet rather than the Asante GigaNIX (only has a beta driver for OS X available). If with this machine they are still dropping frames I'm not sure what to try next. Any suggestions?

The thing to keep in mind here is that this is real-time video capture. It doesn't really matter if I'm able to transfer 50MBps if there are going to be "network hiccups" that are long enough for data not to be written/read when it needs to. I have the feeling it is this unpredictable pause on the network/server that is killing me. Anyway to test, or diminish, these "network hiccups" from occurring?

[kennedy]

Originally posted by weborican:
I havce now replaced the server with a G4 Dual 500 machine with 15GB of RAM and I'm crossing my fingers that it will work. There are two things that are being tested here: 1) Faster and dual processors, 2) Built-in Gigabit Ethernet rather than the Asante GigaNIX (only has a beta driver for OS X available). If with this machine they are still dropping frames I'm not sure what to try next. Any suggestions?

How'd the Dual 500 setup do?

[weborican]

I spoke with the client today and it seems the are STILL having playback and capture problems even with the G4 Dual 500 machine!!! Ugh! Now what?!?!?! I'm flat-out of reasons why this is not working as it should and wondering if this is why people have been waiting for so long for a true OS X SAN solution.

What kills me is that these problems are happening even though there is only one computer reading/writing at that point in time. Since I had 3 computers capturing video for a few minutes without a problem during one of my tests, I'm still convinced bandwidth is not the problem.

What could be causing these network hiccups? I have one of the ports on my Gigabit Switch plugged into the other PC network switch, but that (in theory) shouldn't be a problem...right? Any tools I can run on the network for a day or so to see what is happening on the network at the moment they get a dropped frame? Network sniffer? :-(

[kennedy]

Disclaimer: I don't have any practical experience with the things you're dealing with, such as Final Cut Pro... but since nobody smart has stepped up, I figured I'd throw some ideas out...

Generally speaking, real-time capture implies a dedicated input system that has no other responsibility than that... so it guarantees its always "listening". If that input system is feeding into a non-dedicated system, then it is responsible for providing adequate buffering to overcome the longest "possible" wait times on the non-dedicated system.

In your scenario, the storage system is obviously non-dedicated, as is the network. By its nature, Ethernet will always have hiccups... its designed to allow hiccups and recover. Thus, given a real-time input system feeding through Ethernet to a storage system, it is the *input* system's responsibility to buffer adequately for the *maximum*latency* of the Ethernet. The Ethernet can only be responsible for having adequate *bandwidth* to keep up with the *average* load of all the input systems, plus some *excess* to allow it to catch back up after contentions occur.

Most of your posts have been talking about the non-dedicated portion of the system's bandwidth... and it would seem you have adequate for your needs. But you haven't mentioned anything about the buffering provided by the dedicated portion of your system. And here's where my lack of any knowledge about Final Cut Pro or the real-time video capture system prevents me from doing more than guessing...

I'd be looking at the buffer or cache settings on the real-time capture system. I'd be looking at anything else running on those systems, as well. OS X is NOT a real-time system... its a multi-tasking OS... it is non-dedicated by its nature. What priority do they have the real-time capture processes running at? What else do they have running? Where is the video actually being captured? There must be some hardware doing some buffering feeding into the OS X systems... what is that and how much buffering does it provide?

Note that OS9, with its inferior tasking capabilities, is actually superior for real-time stuff... you can have a task take-over the computer in a somewhat more reliable way.

The final bit of wisdom that I will volunteer is from queueing theory. You need to look at each place where you can encounter latency (contention for the same resource) and examine how far behind you could fall... that tells you min buffer. But THEN you need to look at how much excess bandwidth you have to determine how quickly you will catch up. Based on that, you must look at max frequency of contention and relate that to how long it takes to catch up to see if you may need more buffer to deal with multiple collisions. Then, given multiple levels of latency/buffering, you need to understand how they compound.

The easiest way out of that complexity is to make sure that between your dedicated capture hardware and your first non-dedicated system that you have plenty of buffering to handle all the latencies throughout the rest.

Hopefully that made sense. If not, I'll need a more detailed description of what's really going on the front-side of the network.
I think that's where your problem is. There is nothing you can do on the back side of an Ethernet to eliminate hiccups... Ethernet hiccups by design. You have to manage it on the front-side.