python - Is there a faster pythonic method to read first few lines of file than fid.readline()? -


i have open several thousand files, read first 3 lines.

currently, doing this:

def test_readline(filename):     fid = open(filename, 'rb')     lines = [fid.readline() in range(3)] 

which yields result:

the slowest run took 10.20 times longer fastest. mean intermediate result being cached. 10000 loops, best of 3: 59.2 µs per loop

an alternate solution convert fid list:

def test_list(filename):     fid = open(filename, 'rb')     lines = list(fid)   %timeit test_list(myfile) 

the slowest run took 4.92 times longer fastest. mean intermediate result being cached. 10000 loops, best of 3: 374 µs per loop

yikes!! there faster way read first 3 lines of these files, or readline() best? can respond alternatives , timings please?

but @ end-of-the-day have open thousands of individual files , not cached. thus, matter (looks does)?

(603µs uncached method readline vs. 1840µs list method)

additionally, here readlines() method:

def test_readlines(filename):     fid = open(filename, 'rb')     lines = fid.readlines()      return lines 

the slowest run took 7.17 times longer fastest. mean intermediate result being cached. 10000 loops, best of 3: 334 µs per loop

you can slice iterable itertools.islice:

import itertools   def test_list(filename):     open(filename, 'r', encoding='utf-8') f:         return list(itertools.islice(f, 3)) 

(i changed open bit because it’s unusual read files in binary mode line, can revert that.)


Comments

Popular posts from this blog

What is happening when Matlab is starting a "parallel pool"? -

php - Cannot override Laravel Spark authentication with own implementation -

Qt QGraphicsScene is not accessable from QGraphicsView (on Qt 5.6.1) -