Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > macOS > Confused about line feeds, return key codes etc.

Confused about line feeds, return key codes etc.
Thread Tools
darcybaston
Grizzled Veteran
Join Date: May 2000
Location: ON, Canada
Status: Offline
Reply With Quote
Dec 9, 2003, 03:08 PM
 
This problem began with downloading a MySQL dump from phpmyadmin 2.5.4.All the sql dumps have these weird ^M codes at the end of each line. With an iMac running OS X 10.2, SQL dumps had the 'right' line feeds and you could do a 'more filedump.sql' at the CLI and lines would be spaced properly.

I get this new iBook with 10.3, and all my phpmyadmin dumps now have these ^M characters at the end of each line which is confusing the heck out of mysql when trying to do a load with something like:

mysql -uroot -p databasename < dumpfile.sql

That line works fine with files that have line feeds properly formatted, but returns to the prompt without a single error code and does nothing if the .sql file has those ^M codes in it, which are viewable when you do a 'more dumpfile.sql' and look at it in the terminal.

So what gives? Where are these ^M codes coming from and why are they showing up in Panther and I've NEVER seen them with 10.2? Can I filter those line feeds out? Can I fix files that are malformed like that?

Edit : One way to make a proper mysql dump is to use a backup feature available in the administration panel of phpbb2. Now I wish I knew how come phpbb2 can make sql dumps with proper line feed/newline codes and phpmyadmin can't. Or is it a browser text translation that's taking place while using safari?

Just tried it with Mozilla and it too is making ^M riddled files, so it must be phpmyadmin. Will try their forums too.
Macbook (white glossy) 2.16GHz | 4GB RAM | 7200RPM HD | 10.5.x
     
Arkham_c
Mac Elite
Join Date: Dec 2001
Location: Atlanta, GA, USA
Status: Offline
Reply With Quote
Dec 9, 2003, 03:55 PM
 
^M is a carriage return (ASCII 12). Just for reference, Macintosh text files end lines in a carriage return, UNIX files end lines in a linefeed (ASCII 10), and DOS files end in carriage return + linefeed (ASCII 12 + ASCII 10). If the file looks right other than the ^M characters, I'd guess you're getting DOS-formatted text files.

I don't know the cause, but I can offer a solution. Here is a simple python script that will work with 10.3 or later and convert any file (DOS, Mac, or UNIX) into a file with UNIX feeds.

Code:
#!/usr/bin/env python import sys def convertFile(filepath): inf = open(filepath, "rU") lines = inf.readlines() inf.close() outf = open(filepath, "w") for line in lines: outf.write(line) outf.close() if __name__ == "__main__": convertFile( sys.argv[1] )
Mac Pro 2x 2.66 GHz Dual core, Apple TV 160GB, two Windows XP PCs
     
darcybaston  (op)
Grizzled Veteran
Join Date: May 2000
Location: ON, Canada
Status: Offline
Reply With Quote
Dec 9, 2003, 04:04 PM
 
Hey thanks! That is one very cool solution. I pasted the text to a file in /usr/bin, chmod'ed it to +x and it works GREAT!

Thank you so much. I'll never have that problem again.
Macbook (white glossy) 2.16GHz | 4GB RAM | 7200RPM HD | 10.5.x
     
philzilla
Occasionally Useful
Join Date: Jun 2001
Location: Liverpool, UK
Status: Offline
Reply With Quote
Dec 9, 2003, 05:17 PM
 
i just use this perl one-liner:
Code:
perl -pi -e 's/\r\n?/\n/g' filename
is the output much different to that python script?
"Have sharp knives. Be creative. Cook to music" ~ maxelson
     
darcybaston  (op)
Grizzled Veteran
Join Date: May 2000
Location: ON, Canada
Status: Offline
Reply With Quote
Dec 9, 2003, 05:23 PM
 
Oh hey...it's the Zilla man! Looked for you earlier to see if you had a solution for me heh.

I just tried your suggestion and it works too. Problem is, there's no way I'd remember that one, however an alias could be useful.

Anybody know which solution would perform quicker? I have 60 megabyte sql dumps to filter through.
Macbook (white glossy) 2.16GHz | 4GB RAM | 7200RPM HD | 10.5.x
     
philzilla
Occasionally Useful
Join Date: Jun 2001
Location: Liverpool, UK
Status: Offline
Reply With Quote
Dec 9, 2003, 05:50 PM
 
Originally posted by darcybaston:
Anybody know which solution would perform quicker? I have 60 megabyte sql dumps to filter through.
tried LineBreak?

try backing that 60Mb .sql file and trying the one-liner on it. it's easy enough to make an alias to it.
"Have sharp knives. Be creative. Cook to music" ~ maxelson
     
darcybaston  (op)
Grizzled Veteran
Join Date: May 2000
Location: ON, Canada
Status: Offline
Reply With Quote
Dec 10, 2003, 11:13 AM
 
Performance test results.

I ran the python script and the perl solution on the same 58 megabyte file. I opened two shell windows, typed in the commands and then pressed enter in each window as quickly as I could in each.

Philzilla's perl one liner (which I had aliased) finished about 8 seconds sooner than the python solution (script in a file with +x in /usr/bin). I don't know what priority was given to each solution which may have interfered but from this little sloppy test, perl did it faster.
Macbook (white glossy) 2.16GHz | 4GB RAM | 7200RPM HD | 10.5.x
     
philzilla
Occasionally Useful
Join Date: Jun 2001
Location: Liverpool, UK
Status: Offline
Reply With Quote
Dec 10, 2003, 11:21 AM
 
RAAAWR!!!

glad to be of service, Darcy San
"Have sharp knives. Be creative. Cook to music" ~ maxelson
     
darcybaston  (op)
Grizzled Veteran
Join Date: May 2000
Location: ON, Canada
Status: Offline
Reply With Quote
Dec 10, 2003, 11:31 AM
 
I bow to you both, oh wise masters of the CLI-Click-Clan.
Macbook (white glossy) 2.16GHz | 4GB RAM | 7200RPM HD | 10.5.x
     
Sven G
Professional Poster
Join Date: Dec 2000
Location: Milan, Europe
Status: Offline
Reply With Quote
Dec 10, 2003, 11:39 AM
 
Another line ending conversion utility: mac2unix, available on the Mac GPG web site.

The freedom of all is essential to my freedom. - Mikhail Bakunin
     
drmbb2
Forum Regular
Join Date: Jun 2001
Location: On the move again...
Status: Offline
Reply With Quote
Dec 11, 2003, 02:46 PM
 
There's also strp which has been around for a few years. It compiles and runs fine on OS X.
     
Camelot
Mac Elite
Join Date: May 1999
Location: San Jose, CA
Status: Offline
Reply With Quote
Dec 12, 2003, 01:41 AM
 
Originally posted by darcybaston:
[B][snip] I opened two shell windows, typed in the commands and then pressed enter in each window as quickly as I could in each.[snip]
FWIW, a more precise comparison can be achieved using the 'time' utility:

time <command>

It will run <command> then output the exact time taken to run, indicating CPU time as well as overall process time.
Gods don't kill people - people with Gods kill people.
     
[APi]TheMan
Mac Elite
Join Date: Sep 2001
Location: Chico, CA and Carlsbad, CA.
Status: Offline
Reply With Quote
Dec 15, 2003, 07:42 PM
 
Originally posted by Camelot:
FWIW, a more precise comparison can be achieved using the 'time' utility:

time <command>

It will run <command> then output the exact time taken to run, indicating CPU time as well as overall process time.
Ooh la la! Cool. That's pretty nifty.
"In Nomine Patris, Et Fili, Et Spiritus Sancti"

     
   
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 02:59 AM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,