Update¶
Stable version¶
Using pip
¶
From PyPi¶
$ pip3 install --user --upgrade PyFunceble
Note
We recommend the --user
flag which installs the required dependencies
at the user level. More information about it can be found on
pip documentation.
Warning
We do not recommend the --user
flag when using PyFunceble
into containers or CI engines.
From GitHub¶
$ pip3 install --user --upgrade git+https://github.com/funilrys/PyFunceble.git@master#egg=PyFunceble
Note
We recommend the --user
flag which installs the required dependencies
at the user level. More information about it can be found on
pip documentation.
Warning
We do not recommend the --user
flag when using PyFunceble
into containers or CI engines.
Using the AUR (for Arch Linux users)¶
With makepkg¶
$ curl https://aur.archlinux.org/cgit/aur.git/plain/PKGBUILD?h=python-pyfunceble -o PKGBUILD
$ makepkg
$ sudo pacman -U python-pyfunceble*.tar.xz
With your favorite AUR helper¶
Warning
We do not recommend any AUR helper but keep in mind that some AUR helpers are “better” than other. For more information about your current (or any other) AUR helper please report to the ArchWiki page.
$ yourFavoriteAurHelper -Syu python-pyfunceble
Using docker (hub)¶
The image description can be found at https://hub.docker.com/r/pyfunceble/pyfunceble
$ docker pull pyfunceble/pyfunceble
Pure Python method¶
Execute the following and enjoy PyFunceble!
$ cd PyFunceble
$ git checkout master && git fetch origin && git merge origin/master
$ python3 setup.py test
$ python3 setup.py install # Avoid this if you want to uninstall one day.
$ pip3 install --user --upgrade -e . # Prefer this method.
Note
We recommend the --user
flag which installs the required dependencies
at the user level. More information about it can be found on
pip documentation.
Warning
We do not recommend the --user
flag when using PyFunceble
into containers or CI engines.
Development version¶
For development¶
$ cd PyFunceble && git checkout dev
$ git fetch origin && git merge origin/dev
Note
As you previously installed with
Only code/repository update is required.
For usage¶
Using pip
¶
From PyPi¶
$ pip3 install --user --upgrade PyFunceble-dev
Note
We recommend the --user
flag which installs the required dependencies
at the user level. More information about it can be found on
pip documentation.
Warning
We do not recommend the --user
flag when using PyFunceble
into containers or CI engines.
From GitHub¶
$ pip3 install --user --upgrade git+https://github.com/funilrys/PyFunceble.git@dev#egg=PyFunceble
Note
We recommend the --user
flag which installs the required dependencies
at the user level. More information about it can be found on
pip documentation.
Warning
We do not recommend the --user
flag when using PyFunceble
into containers or CI engines.
Using the AUR (for Arch Linux users)¶
With makepkg¶
$ curl https://aur.archlinux.org/cgit/aur.git/plain/PKGBUILD?h=python-pyfunceble-dev -o PKGBUILD
$ makepkg
$ sudo pacman -U python-pyfunceble-dev*.tar.xz
With your favorite AUR helper¶
Warning
We do not recommend any AUR helper but keep in mind that some AUR helpers are “better” than other. For more information about your current (or any other) AUR helper please report to the ArchWiki page.
$ yourFavoriteAurHelper -Syu python-pyfunceble-dev
Using docker (hub)¶
The image description can be found at https://hub.docker.com/r/pyfunceble/pyfunceble-dev
$ docker pull pyfunceble/pyfunceble-dev
Pure Python method¶
Execute the following and enjoy PyFunceble!
$ cd PyFunceble && git checkout dev
$ git fetch origin && git merge origin/dev
$ python3 setup.py test
$ python3 setup.py install # Avoid this if you want to uninstall one day.
$ pip3 install --user --upgrade -e . # Prefer this method.
Note
We recommend the --user
flag which installs the required dependencies
at the user level. More information about it can be found on
pip documentation.
Warning
We do not recommend the --user
flag when using PyFunceble
into containers or CI engines.
Important information for pyfunceble >= 4.0.0
¶
When you upgrade PyFunceble from <4.x
to any version 4.x
there will be:
a SQL conversion if you use the
--database-type
mysql
ormariadb
.a JSON to CSV conversion if one of those files is found in your filesystem:
inactive_db.json
whois_db.json
The conversion of both SQL and json to CSV will take a “bit” of time as, it is done in a single process mode, to avoid any hick-ups instead of loading the entire file into memory. Loading the entire
*.json
file into memory can have severe consequences depending on the size of the source file.Note
Once the job is done the
json2csv
shouldn’t appear again for this or later4.x
releases.A workaround for waiting on the rather slow
json2csv
, you can deleteinactive_db.json
whois_db.json
(Very Very Very bad idea in the long run…)
However you will probably not benefit for by deleting the
whois_db.json
as this is the definitive slowest lookup process in the test flow, do to the limitation in available API call you can do toWHOIS
servers before getting banned. Therefore we CAN NOT recommend deleting this file, rather than waiting for the conversion to finish.The output directory structure have been altered to work with the ability to test more than one source at the time. Prior to version
4.0.0.ax
the output hierarchy looked likeoutput/domains/ACTIVE/list
. In Pyfunceble version>=4.x
this have been altered to include the source name and append to the folder structure. From this version it will therefor looks likeoutput/{{ input_source_name }}/domains/ACTIVE list
.
Note
As consequence of the time consuming conversion, we will advise you to run a simple pyfunceble command like:
pyfunceble -d mypdns.org
pyfunceble --database-type mariadb -d mypdns.org
How long time does it take¶
A few numbers to help you schedule your upgrade process.
We have tested the SQL conversion with the following specifications
- 2 Xeon CPU x86_64 (8 cores) 2 GHz
- 48 GB ram.
- 1 SSD Kingston KC-600
- Mariadb 10.5, default config
- Non dedicated
- The database contains
- Roughly
265.000
records in the test tables - Approximately
1.000.000
records within thepyfunceble_whois
table.
- Roughly
This process toke about 10 hours to complete.
Important information for >= 3.2.11
¶
When you update from dev@<=3.2.10 or master@<=3.2.2 to newer release, there will be made a SQL conversion of the databases table layout. This can take up a sagnificent amount of time based on the size of the database.
The table layout converion is being made to:
- Minimize the total size
- Optimize the sql flow and minimizing the read/write to save disk I/O.
- Minimize the number of SQL queries being made
It have been seen taking days to convert these tables on very large installations.