[l008com]

[Edit: Subject Changed]
I'm trying to write a script thats going to be part of the back-end of one of my web sites. Now I can use curl on this machine no problem, to download all 250 MB of the file I need it to get. and i can wget to get it too btw. But with either command, if I call it from within a php shell script, it downloads about the first 20-25% of the 250MB file, then it stops. I can't figure out for the life of me, why it keeps stopping like this?

BTW I tried both shell_exec() and using the command in `these` tick mark things. No go

[VEGAN]

why do you even use `curl'?

PHP comes with ftp functions [if they are enabled at compile time]

or even remote file options:
http://www.php.net/manual/en/features.remote-files.php

you are trying maybe too hard

[l008com]

Because I'm not reading this file into a buffer, its 250 MB, and its gzipped, i need to download it all to the desktop, then unzip it. curl seemed like the best way.

[Arkham_c]

Originally posted by l008com:
Because I'm not reading this file into a buffer, its 250 MB, and its gzipped, i need to download it all to the desktop, then unzip it. curl seemed like the best way.

Might I suggest that PHP may not be the best tool for dealing with this? I know some people would argue that PHP is a peer to perl, python, or java, but I would not. PHP is fine for web apps, but downloading 250MB files is reall out of its realm. Either write a pure shell script to do what you want, or if that is insufficient, consider using one of these more capable languages.

[l008com]

Originally posted by Arkham_c:
Might I suggest that PHP may not be the best tool for dealing with this? I know some people would argue that PHP is a peer to perl, python, or java, but I would not. PHP is fine for web apps, but downloading 250MB files is reall out of its realm. Either write a pure shell script to do what you want, or if that is insufficient, consider using one of these more capable languages.

I solved the problem, I just had to set the timeout to 0, which turns it off and the script will run forever. As far as php being right for the task, its perfect. Its probably the best choice. This script is scanning through a very very large text file, and taking certain items from the file, and inserting them into a MySQL database. Its the perfect job for php, it just has to download and unzip the file before it can start scanning it. PHP is startlingly fast too by the way, its running on a nearly stock 8500 with 10.1 crammed onto it, and its FLYING!

[l008com]

HEE HEE!! Instead of starting a new thread, I'll just change this one into what I need next. I can't install MySQL!! I downloaded the source. I did ./configure and everything seemed to work out alright. But when I do make, it gets all kinds of errors I don't understand. I know they make packages, but all are for 10.2 and up. And I can only run 10.1 on this machine (its a 604) Also, I really need MySQL 4.1 (not 4.0) because I need one of the elements that are new to it. If anybody has a clue, I'd appreciate it, because stuff like this is waay over my head!

depmode=gcc /bin/sh ../../depcomp \
cc -DUNDEF_THREADS_HACK -DHAVE_CONFIG_H -DNO_KILL_INTR -I. -I. -I../.. -I../../include -I./../.. -I.. -O -DDBUG_OFF -traditional-cpp -DHAVE_DARWIN_THREADS -D_P1003_1B_VISIBLE -DSIGNAL_WITH_VIO_CLOSE -DSIGNALS_DONT_BREAK_READ -DHAVE_BROKEN_REALPATH -DFN_NO_CASE_SENCE -c `test -f chared.c || echo './'`chared.c
In file included from compat.h:24,
from chared.c:39:
strlcpy.h:1: parse error before `strlcpy'
strlcpy.h:1: parse error before `size_t'
strlcpy.h:1: warning: data definition has no type or storage class
strlcpy.h:2: parse error before `strlcat'
strlcpy.h:2: parse error before `size_t'
strlcpy.h:2: warning: data definition has no type or storage class
make[3]: *** [chared.o] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

[lars-man]

Originally posted by l008com:
I solved the problem, I just had to set the timeout to 0, which turns it off and the script will run forever.

This is probably not such a good idea. What is going to happen if one of your scripts ends in an infinite loop and/or consumes a lot of CPU time or something like that? That could screw up your httpd...

Kind regards
Lars

[theolein]

Originally posted by l008com:
HEE HEE!! Instead of starting a new thread, I'll just change this one into what I need next. I can't install MySQL!! I downloaded the source. I did ./configure and everything seemed to work out alright. But when I do make, it gets all kinds of errors I don't understand. I know they make packages, but all are for 10.2 and up. And I can only run 10.1 on this machine (its a 604) Also, I really need MySQL 4.1 (not 4.0) because I need one of the elements that are new to it. If anybody has a clue, I'd appreciate it, because stuff like this is waay over my head!

First your php script: Yes, that's what I thought as well, i.e. that your script was timing out, but it means that you're running it as a web application and not as a shell script. I assume that you have a web page that downloads a file onto the server itself. The warnings that the others have given you are real and you should watch out for them. PHP is riddled with security holes and if your script is not written with security in mind someone could exploit your script and download some monstrous file onto your server, making it unbootable. Either that or they could exploit your shell access code to execute some code on the server. Be very, very careful! (SSL, Passwords etc and don't use http get requests for a start)

Secondly: Save yourself the trouble of compiling Mysql and simply get a built package with instructions here.