[adamk]

hi,

i am working on a pure-java program that does some statistical analysis on some randomly-generated datasets and am wondering if threading would be a good idea for the program. as it stands right now, all the creation of the random datasets (user-defined ~100) and subsequent analysis are done sequentially along with writing of the original data and results.

i was thinking of creating a class that would handle all the functions (creation + writing, analysis + writing) as a thread and starting the threads at some increment, 0.1s, apart.

what benefits, if any, would i gain from changing my current program design? here are some concerns,
-the program is run only on single-processor comps now, but may be used on dual in the future,
-would threads and the writing of files slow down execution?
-the program runs fine as is, processing 100 sets takes 2-3 minutes on average, which is a lot faster than the previous times on the order of hours. would threading improve this? i doubt since most of the CPU is used by the program.

also, for longer datasets i use long instead of ints and for my pismo, takes a lot longer to process these longer datasets. if i had a 64-bit processor, would the data be processed as quickly as a 32-bit int?

thanks, adam

[larkost]

Threading usually slows down the execution of a program, unless you either have more processors to go to, or you have a lot of time-outs in the program (waiting for network, waiting for disk, etc... but even then all of that becomes complicated). Threading also creates a lot of complexity in a program, and it can be easy to get into contention situations (which can slow down a program, or even stop it altogether).

So the chances are that a single threaded program is probably the better way. I would spend my time on trying to optimize rather than thread.

On the 64bit thing. Yes, having hardware that can process 64bit math in a single go (rather than emulating it with multiple cycles) can dramatically speed things up. I am not sure how Java currently handles 64bit chips, so there is a chance that this hardware would be ignored anyways... but I am worry-warting there. It would probably improve things dramatically.

[adamk]

Originally posted by larkost:
Threading usually slows down the execution of a program, unless you either have more processors to go to, or you have a lot of time-outs in the program (waiting for network, waiting for disk, etc... but even then all of that becomes complicated). Threading also creates a lot of complexity in a program, and it can be easy to get into contention situations (which can slow down a program, or even stop it altogether).

So the chances are that a single threaded program is probably the better way. I would spend my time on trying to optimize rather than thread.

On the 64bit thing. Yes, having hardware that can process 64bit math in a single go (rather than emulating it with multiple cycles) can dramatically speed things up. I am not sure how Java currently handles 64bit chips, so there is a chance that this hardware would be ignored anyways... but I am worry-warting there. It would probably improve things dramatically.

thanks for the reply. i really wanted to try and learn threads, but it may not be the best solution in this case. while thinking more about it, i got the idea of placing threads in series - starting the subsequent thread after the previous thread is, say, 90% complete. in this way, the second thread could start processing while the first waits for the system to write the results to disk.

i would eventually like the design to be able to produce quicker results on a cluster or multiple CPU's.

adam

[BLAZE_MkIV]

What you should probably have then is one thread reading and writing your data to the disk and another performing the analysis. That way the file io thread can keep one "block" ahead of the analysis thread and you don't have to put any file io login into the analysis portion. this way you could also test for multiple cpus and spawn more analysis threads if nessicary.