Christof Buchbender
Hosted also here: https://hera.ph1.uni-koeln.de/~buchbend/python_best_practices.html
Although python2.7 is still widely used and the default on many distributions, python3 will become the standard in the near future. Important packages such as astropy do not release new versions for python2 any more. So if you just start to use python or do not have legacy code that needs python2 you should switch to python3 now. Good references are:
No matter which python version you use, using virtual environments facilitates the work with python. A virtual environment is a container into which all packages and dependencies are installed. One can setup multiple virtual environments in parallel and switch between them. Personally I am using the helper package virtualenvwrapper to maintain my virtual environments setup with virtualenv both for python2 and python3. However, in python3 a native implementation of virtual environments check https://docs.python.org/3/library/venv.html for the details. If you only use python3 this is the recommended way to employ virtual environments.
Even single scripts that are longer than just a few lines benefit from being packaged as a proper python package. Packages have the following benefits:
from setuptools import setup
setup(name='kosma_update_fits_header',
version="1.0",
description=('Tool to update the fits header'),
include_package_data=True,
packages=['kosma_update_fits_header'],
url='',
author='Christof Buchbender',
author_email='buchbend@ph1.uni-koeln.de',
license='MIT',
install_requires=['pandas',
'astropy',
'argcomplete',
'pyyaml'],
entry_points={"console_scripts": [
('kosma_update_fits_header = '
'kosma_update_fits_header.kosma_update_fits_header:main'),
('remove_all_emu_scans = '
'kosma_update_fits_header.remove_all_emu_scans:main'),
('fits_file_info = '
'kosma_update_fits_header.find_and_filter_fits_files:main'),
('fits_check_for_simulator = '
'kosma_update_fits_header.find_and_filter_fits_files:'
'check_for_simulation_scans'),
('fits_correct_pointing_model = '
'kosma_update_fits_header.kosma_update_fits_header:'
'correct_pointing_model_main'),
('get_scan_range = '
'kosma_update_fits_header.find_and_filter_fits_files:'
'get_scan_range')
]
},
zip_safe=False)
The coding style guidelines for python that are defined in the PEP8 document
are a de-facto standard in the python community. Following these coding style guide
makes your program easily readable by other Python developers and lead to a consistent
programming style. One can use linter programs that check wether your codes complies
with the guidelines.
One of the most useful rules is IMHO to not use lines that are longer than 79 or 80 characters. This accomplishes two things. One can open two or three scripts side by side on a normal Full-HD screen. The second thing is that one is forced to write code that is easier to read. In python one can make pretty incomprehensible lines of code by nesting functions into each other.
Try to make functions and classes as concise as possible. One can create Objects or functions in python that do it all easily. But in the long run this makes debugging the function or Class very difficult. One should use rather many small objects or classes with very consise tasks and approriate names.
Python comes with a very flexible package for logging:
The logging system has different warning levels and can be configured to write to screen, to files or other output streams. This way it is easy to write out consistent debugging messages from your program. For example one can list always the file and line number or the function name in the debug message:
import logging
logging.warning("This is a simple Python logger example")
WARNING:root:This is a simple Python logger example#importing module
import logging
###
# Next two lines only needed for jupyter notebooks
from importlib import reload
reload(logging)
###
#Create and configure logger
logging.basicConfig(format='%(asctime)s %(name)s %(levelname)s %(message)s')
#Creating an object
logger=logging.getLogger()
#Setting the threshold of logger to DEBUG
logger.setLevel(logging.DEBUG)
#Test messages
logger.debug("Debug Message")
logger.info("Information")
logger.warning("Warning")
logger.error("Error")
logger.critical("Critical")
###
# Next two lines only needed for jupyter notebooks
from importlib import reload
reload(logging)
###
logging.basicConfig(filename="newfile.log", format='%(asctime)s %(name)s %(levelname)s %(message)s')
logger.debug("Debug Message")
logger.info("Information")
logger.warning("Warning")
logger.error("Error")
logger.critical("Critical")
Python is an object oriented language, but one can get around without ever designing own classes. Classes are not always nessecary but sometimes they are superior to functions and more convenient to use.
class Example(object)class Vector(object):
def __init__(self, x, y):
self.x = x
self.y = y
def __add__(self, other):
return Vector(
x=self.x+other.x,
y=self.y+other.y
)
def __mul__(self, other):
return self.x * other.x + self.y + other.y
def __str__(self):
return "<{},{}>".format(self.x, self.y)
def __repr__(self):
return "<{},{}>".format(self.x, self.y)
a = Vector(1, 2)
b = Vector(3, 4)
print("a+b =", a+b)
print("a*b =", a*b)
type(9)
Use e.g. git to version control your programs:
This is a guideline on how to write good programs in general not just python. It is a good reference for best practices for software development.
basic tools:
databases:
distributed/paralell computing
machine learning:
web development (e.g. for data presentation):
miscellaneous:
webscraping (gathering data online):
NOT PYTHON:
There are many introduction books and tutorials the books I chose here go deeper than typical introductions.