| python Programming Glossary: diskPython Numpy Very Large Matrices http://stackoverflow.com/questions/1053928/python-numpy-very-large-matrices  numpy are the way to go. PyTables will store the data on disk in HDF format with optional compression. My datasets often get.. 
 “Large data” work flows using pandas http://stackoverflow.com/questions/14262433/large-data-work-flows-using-pandas  first thought is to use HDFStore to hold large datasets on disk and pull only the pieces I need into dataframes for analysis... the following Loading flat files into a permanent on disk database structure Querying that database to retrieve data to.. import a large flat file and store it in a permanent on disk database structure. These files are typically too large to fit.. 
 How do I watch a file for changes using Python? http://stackoverflow.com/questions/182197/how-do-i-watch-a-file-for-changes-using-python  'hear' any updates to the file the way it does on a local disk.  python file pywin32 watch   share improve this question  .. 
 Simple Python Challenge: Fastest Bitwise XOR on Data Buffers http://stackoverflow.com/questions/2119761/simple-python-challenge-fastest-bitwise-xor-on-data-buffers  is a bit slower since the code needs to be loaded from the disk and cached subsequent invocations are faster import numpy import.. 
 when does Python allocate new memory for identical strings? http://stackoverflow.com/questions/2123925/when-does-python-allocate-new-memory-for-identical-strings  ... is stored only once fine. BUT write the list to disk and read it back in again the same list now has N different.. 
 Solving embarassingly parallel problems using Python multiprocessing http://stackoverflow.com/questions/2359253/solving-embarassingly-parallel-problems-using-python-multiprocessing  putting data in the # input queue. # Write the results to disk as soon as they appear on the output # queue. # Ensure all child.. 
 Python snippet to remove C and C++ comments http://stackoverflow.com/questions/241327/python-snippet-to-remove-c-and-c-comments  the comments removed. Of course if you have the file on disk you could have the input and output variables be file handles.. link and it should be saved in a readable location on disk. sed is also available on Windows and comes installed by default.. 
 Create directory if it doesn't exist for file write http://stackoverflow.com/questions/273192/create-directory-if-it-doesnt-exist-for-file-write  due to other factors such as insufficient permissions full disk etc. One option would be to trap the OSError and examine the.. 
 How to get current CPU and RAM usage in Python? http://stackoverflow.com/questions/276052/how-to-get-current-cpu-and-ram-usage-in-python  way of getting current system status current CPU RAM free disk space etc. in Python Bonus points for nix and Windows platforms... 
 Inverse Distance Weighted (IDW) Interpolation with Python http://stackoverflow.com/questions/3104781/inverse-distance-weighted-idw-interpolation-with-python  a Python solution which doesn't require multiple writes to disk and launching external processes. The number of calls to the.. 
 How should I log while using multiprocessing in Python? http://stackoverflow.com/questions/641420/how-should-i-log-while-using-multiprocessing-in-python  such that its log goes to a different file descriptor to disk or to pipe. Ideally all log entries should be timestamped. Your.. be timestamped. Your controller process can then if using disk files coalesce the log files at the end of the run sorting by.. 
 Generating file to download with Django http://stackoverflow.com/questions/908258/generating-file-to-download-with-django  myfile.zip' return response If you don't want the file on disk you need to use StringIO import cStringIO as StringIO myfile.. 
 Why is reading lines from stdin much slower in C++ than Python? http://stackoverflow.com/questions/9371238/why-is-reading-lines-from-stdin-much-slower-in-c-than-python  20M lines of text. Yes I ran it several times to eliminate disk caching confound. usr bin time cat test_lines_double . readline_test_cpp.. file 100M lines ~3.4GB on a fast server with very fast disk comparing the python the unsynced cin and the fgets approaches.. C code. Again this is for a 100M line file on a fast disk. Here's the complete table now Implementation Lines per second.. 
 How do you create an osx application/dmg from a python package? http://stackoverflow.com/questions/116657/how-do-you-create-an-osx-application-dmg-from-a-python-package  FileStorm Guides to doing it manually How to Create a Disk Image Installer for Apple Mac OS X How to create a custom dmg.. 
 Differences between Framework and non-Framework builds of Python on Mac OS X http://stackoverflow.com/questions/1444543/differences-between-framework-and-non-framework-builds-of-python-on-mac-os-x  would download and install the Python 2.6.2 Mac Installer Disk Image which is a framework build and go about my business using.. 
 Python Disk-Based Dictionary http://stackoverflow.com/questions/226693/python-disk-based-dictionary  Disk Based Dictionary  I was running some dynamic programming code.. 
 Why doesn't the save button work on a matplotlib plot? http://stackoverflow.com/questions/3692928/why-doesnt-the-save-button-work-on-a-matplotlib-plot  python.org's Python 2.7 PPC i386 x86 64 Mac OS X Installer Disk Image for Mac OS X 10.5 . Thanks to Ned Deily's answer to this.. Python 2.7 using the Python 2.7 32 bit Mac OS X Installer Disk Image for Mac OS X 10.3 through 10.6 Is there a different gui_support.. 
 Get hard disk serial number using Python on Linux http://stackoverflow.com/questions/4193514/get-hard-disk-serial-number-using-python-on-linux  fields 10 .strip model fields 15 .strip print Hard Disk Model s model print Serial Number s serial_no ## For documentation.. laptop ~ Desktop hdserial sudo python hd.py dev sda Hard Disk Model FUJITSU MHV2080BH PL Serial Number NW What's going on.. 
 Sqlite3, OperationalError: unable to open database file http://stackoverflow.com/questions/4636970/sqlite3-operationalerror-unable-to-open-database-file  you're likely to have permissions ownership problems. Disk space is another serious gotcha but less likely. I don't think.. 
 Is “with” monadic? http://stackoverflow.com/questions/7131027/is-with-monadic  any failure in the I O can be hidden from the code. Disk corruption the absence of the named file and an empty file can.. 
 |