programming

Python

Environments

virualenv

it is limited to the python version prsent on the host linux distribution. it handles the pip package manager.

anaconda

anaconda allows to get multiple version of python even if the host linux distribution does not. it handles both conda and pip package managers, but conda is the prefered way.

nix

nix proposes an alternate way of packaging python environments. It is not compatible with pip.

guix

it has less packages than nix and has the same drawbacks: python packages has to be packaged the guix way to be accessible.

Data

Pandas

Koalas

Map/Reduce

Dask

Dask peut cohabiter avec spark sur un cluster hadoop via dask-yarn. Il peut lire/ecrire sur hdfs des fichiers parquet/orc.

Le fonctionnement est largement inspiré de spark: dask s'appuie sur la résolution d'un DAG lazy, avec possibilité de caching des étapes.

La finalité de dask est de transformer des big-data en small-data pour pouvoir les traiter avec pandas.

API

Flask

Dashboard

Dash

Database

ZODB

ZODB uses “object traversal” to retrieve information from the persistance layer. It supports ACID transaction and is ideal for complex datamodel.

Compilation

numba

numba

cython

This page was last modified: