Usage¶
PyFunceble Python API¶
If you are working with a python script, module or even class, you can integrate PyFunceble to your main logic by importing it and using its API (cf: API Documentation).
This section will present some example of the way you can interact with PyFunceble from anything written in Python.
Check the availability of a domain¶
from PyFunceble import DomainAvailabilityChecker
checker = DomainAvailabilityChecker()
to_test = "github.com"
# You can do it this way.
status = checker.set_subject(to_test).get_status()
# Or this way.
checker.set_subject(to_test)
status = checker.get_status()
# We can convert the status to json.
status_json = status.to_json()
# We can convert the status to dict.
status_dict = status.to_dict()
# We can ask "questions".
print(f"Is {to_test} ACTIVE ?", status.is_active())
print(f"Is {to_test} INACTIVE ?", status.is_inactive())
print(f"Is {to_test} INVALID ?", status.is_invalid())
Check the availability of an IP¶
from PyFunceble import IPAvailabilityChecker
checker = IPAvailabilityChecker()
to_test = "192.0.2.1"
# You can do it this way.
status = checker.set_subject(to_test).get_status()
# Or this way.
checker.set_subject(to_test)
status = checker.get_status()
# We can convert the status to json.
status_json = status.to_json()
# We can convert the status to dict.
status_dict = status.to_dict()
# We can ask "questions".
print(f"Is {to_test} ACTIVE ?", status.is_active())
print(f"Is {to_test} INACTIVE ?", status.is_inactive())
print(f"Is {to_test} INVALID ?", status.is_invalid())
Check the availability of an IP or domain¶
from PyFunceble import DomainAndIPAvailabilityChecker
checker = DomainAndIPAvailabilityChecker()
to_test = "github.com"
# You can do it this way.
status = checker.set_subject(to_test).get_status()
# Or this way.
checker.set_subject(to_test)
status = checker.get_status()
# We can convert the status to json.
status_json = status.to_json()
# We can convert the status to dict.
status_dict = status.to_dict()
# We can ask "questions".
print(f"Is {to_test} ACTIVE ?", status.is_active())
print(f"Is {to_test} INACTIVE ?", status.is_inactive())
print(f"Is {to_test} INVALID ?", status.is_invalid())
Check the availability of URL¶
from PyFunceble import URLAvailabilityChecker
checker = URLAvailabilityChecker()
to_test = "http://github.com/PyFunceble"
# You can do it this way.
status = checker.set_subject(to_test).get_status()
# Or this way.
checker.set_subject(to_test)
status = checker.get_status()
# We can convert the status to json.
status_json = status.to_json()
# We can convert the status to dict.
status_dict = status.to_dict()
# We can ask "questions".
print(f"Is {to_test} ACTIVE ?", status.is_active())
print(f"Is {to_test} INACTIVE ?", status.is_inactive())
print(f"Is {to_test} INVALID ?", status.is_invalid())
Check the syntax of domains¶
from PyFunceble import DomainSyntaxChecker
checker = DomainSyntaxChecker()
to_test = "github.com"
# You can do it this way.
status = checker.set_subject(to_test).get_status()
# Or this way.
checker.set_subject(to_test)
status = checker.get_status()
# We can convert the status to json.
status_json = status.to_json()
# We can convert the status to dict.
status_dict = status.to_dict()
# We can ask "questions".
print(f"Is {to_test} VALID ?", status.is_valid())
print(f"Is {to_test} INVALID ?", status.is_invalid())
Check the syntax of IP (v4 or v6)¶
from PyFunceble import IPSyntaxChecker
checker = IPSyntaxChecker()
to_test = "192.0.2.1"
# You can do it this way.
status = checker.set_subject(to_test).get_status()
# Or this way.
checker.set_subject(to_test)
status = checker.get_status()
# We can convert the status to json.
status_json = status.to_json()
# We can convert the status to dict.
status_dict = status.to_dict()
# We can ask "questions".
print(f"Is {to_test} VALID ?", status.is_valid())
print(f"Is {to_test} INVALID ?", status.is_invalid())
Check the syntax of URLs¶
from PyFunceble import URLSyntaxChecker
checker = URLSyntaxChecker()
to_test = "https://github.com/PyFunceble"
# You can do it this way.
status = checker.set_subject(to_test).get_status()
# Or this way.
checker.set_subject(to_test)
status = checker.get_status()
# We can convert the status to json.
status_json = status.to_json()
# We can convert the status to dict.
status_dict = status.to_dict()
# We can ask "questions".
print(f"Is {to_test} VALID ?", status.is_valid())
print(f"Is {to_test} INVALID ?", status.is_invalid())
From a terminal¶
This chapter also relates to writing scripts in bash and PowerShell as the uses the same syntaxes.
--show-completion "shell"
¶
Show shell completion script and exit.
Available Values: bash
, zsh
Note
This argument provides the autocompletion script that you can use to get access to the autocompletion assistance.
It is meant to be used like this:
$ source <(pyfunceble --show-completion bash)
$ pyfunceble --do[TAB]
--domain --dots
--help
¶
Show the help message and exit.
-v
| --version
¶
Show the version of PyFunceble and exit.
Test sources¶
-d "$DOMAIN"
| --domain "$DOMAIN"
¶
This argument takes one or more values separated by spaces.
Test one or more :code:`$DOMAIN`s,
$ PyFunceble -d example.org example.net
A domain is defined as it do NOT start with a protocol://
and it do not
contain a forward slash /
This is a domain: example.org
Note
When this option is used, no output files are generated.
-u "$URI"
| --url "$URI"
¶
Test one or more full URL, separated by spaces.
This argument takes one or more values.
$ PyFunceble --url https://example.org/AlIvE https://example.com/GoNe
Note
When we test the availability of a URL, we (only) check the HTTP status code of the given URL.
A URI or URL is defined by it is starting with a protocol.
This is a URL
http://example.org/?example=yes
This is another URL
https://example.org
This is another URL
ftp://ftp.example.org
-f "$DOMAIN"
| --file "$DOMAIN_FILE"
¶
Read a local or remote (RAW link) file and test all domains inside it. If remote (RAW link) file is given, PyFunceble will download it, and test the content of the given RAW link as if it was a locally stored file.
$ PyFunceble -f "$DOMAIN"_1 "$DOMAIN"_2
$ PyFunceble -f "$DOMAIN_FILE"_1 "$DOMAIN_FILE"_2
$ PyFunceble --file "$DOMAIN_FILE"_1 "$DOMAIN_FILE"_2
Note
- This argument takes one or more space separated values.
- You can combine
-f
and-uf
in the same test.
Warning
You can not combine the usage of -f
, -uf
with
--adblock
at the same time
-uf "$URL_FILES"
| --url-file "$URL_FILES"
¶
Read a local or remote (RAW link) file and test all (full) URLs inside it. If remote (RAW link) file is given, PyFunceble will download it, and test the content of the given RAW link as if it was a locally stored file.
This argument test if a URL which is inside the given file is available. It ONLY tests full URLs.
$ PyFunceble -uf "$URI"_1 "$URI"_2
When a remote located source is provided, we will download the given URL and test its content assuming that each line represents a URL to test.
$ PyFunceble -uf "$URL_FILES"
Note
- This argument takes one or more space separated values.
- You can combine
-f
and-uf
in the same test. - We consider one line as one URL to test.
Warning
A test with this argument consists of the comparison of the status code. No WHOIS record will be requested nor DNS Lookup will be done.
You can not combine the usage of -f
, -uf
and
--adblock
at the same time
Source filtering, decoding, conversion and expansion¶
--adblock
¶
This feature is used to extract all domains, IPv4 and IPv6 addresses from a adblock formatted file and test the status and validate the extracted domains.
To use this feature you’ll need to set the -f "$DOMAIN" | --file "$DOMAIN_FILE" to tell PyFunceble from where to obtain the given list.
$ PyFunceble --adblock -f "$ADBLOCK_FILES"
Default value: adblock: False
Warning
You can not combine the usage of -f
, -uf
and
--adblock
at the same time
--complements
¶
A complement is for example example.org
if www.example.org
is given and vice-versa.
Default value: complements: False
--filter "RegEx"
¶
A Regex string to match in order to test a given line.
Default value: file_filter: null
If you only want to test all blogspot
URI or domains from your list,
this argument allows you to do that!
$ PyFunceble --filter '^\.blogspot\.(com|net)$' -f $DOMAIN_FILE
--mining
¶
Todo
Find out more about how this actually works…
Want to find domain or URL linked to a domain in your list? This argument will exactly do that.
Default value: mining: False
--rpz
¶
New in version 3.3.3.
Activates or disables the decoding of RPZ policies from each given input source
(-f
).
$ PyFunceble --rpz -f $RPZ_FILES
The --rpz
is used to test domains from a fully functional and valid
RPZ (Response Policy Zone). If you do provide the required zone SOA
record it will extract the right domains to test.
Example of a fully functional RPZ zone
spyware.my-rpz.internal. 86400 IN SOA my.awesome.rps.zone. need.to.know.only. 2021011401 300 60 604800 3600
*.360.com.spyware.my-rpz.internal. 86400 IN CNAME .
*.360safe.com.cn.spyware.my-rpz.internal. 86400 IN CNAME .
*.360totalsecurity.com.spyware.my-rpz.internal. 86400 IN CNAME .
360.com.spyware.mypdns.cloud. 86400 IN CNAME .
360safe.com.cn.spyware.mypdns.cloud. 86400 IN CNAME .
360totalsecurity.com.spyware.mypdns.cloud. 86400 IN CNAME .
(PS. RPZ zones does not requires the NS records RFC 1034)
From the example above PyFunceble will be testing the following domains.
You can make a simple test with the above zone example by copy/pasting.
In case your RPZ zone are missing the required SOA
entry, you should
consider combining the --rpz
with –wildcard to
avoid all your wildcard’s domain becoming marked as INVALID
Default value: rpz: False
Warning
You can currently not use the --rpz
in combination with
--syntax
to validate or syntax test a rpz formatted file.
See also
See discussions 149 for more information and participate in it’s development.
--wildcard
¶
New in version 3.3.0.
The flag to use when your source(-f
) of domains starts with a wildcard.
This flag will subtract the *.$DOMAIN
and test the $DOMAIN
according to the test arguments given.
Default value: wildcard: False
As examples of when to use this argument. The first one will
return INVALID if --wildcard
is not set to true.
This feature is related to the –rpz
'*.example.org'
'example.org'
Test control¶
--chancy
¶
New in version 4.1.0b4.dev.
Activates a chancy mode that unleashes the safety workflow in place.
Default value: chancy_tester: False
Warning
You shouldn’t have to use this unless you feel really lucky and trust your machine.
This mode makes things look ‘fast’, but it may produce some unexpected
results if N
process simultaneously write the same output file.
This mode makes the graphical CLI output unparsable - either.
MAY THE FORCE BE WITH YOU!
--cooldown-time
¶
Sets a cooldown time (in second) to be applied between (sleep) before/between each test cycles is done.
Default value: cooldown_time: 0.0
--local
¶
Activates or disables the consideration of the test(s) in or for a local or private network context.
This option is to include testing ability of records only availeble for local and private network. This argument will enable this functionality by removing the usual limitation which do not apply to intranets.
Default value: local_network: False
--collection-preferred-origin
¶
New in version 4.0.0.
Sets the preferred status origin.
Default value: collection.preferred_status_origin: frequent
Available values: frequent
, latest
, recommended
--collection-lookup
¶
New in version 4.0.0.
Activates or disables the usage of the collection lookup whether possible.
Default value: lookup.collection: False
Want to take advantage of the collection API ? This argument is for you.
--collection-lookup-only
¶
New in version 4.1.0b10.
Activates or disables the exclusive usage of the collection lookup as test method.
--dns-lookup
¶
Activates or disables the usage of the DNS lookup whether possible.
Default value: dns: True
Don’t want to perform some DNS lookup? This argument is for you.
--dns-lookup-only
¶
New in version 4.1.0b10.
Activates or disables the exclusive usage of the DNS lookup as test method.
--http-status-code-lookup
| --http
¶
Don’t want to take the result of the HTTP code execution into consideration?
This argument allows you to disable the HTTP status code checker!
Default value: http_status_code: True
--http-status-code-lookup-only
¶
New in version 4.1.0b10.
Activates or disables the exclusive usage of the HTTP Status code lookup as test method.
--netinfo-lookup
¶
Activates or disables the usage of the network information (or network socket) whether possible.
Don’t want to perform some netinfo lookup ? This argument is for you.
Default value: netinfo: True
--netinfo-lookup-only
¶
New in version 4.1.0b10.
Activates or disables the exclusive usage of the NETINFO lookup as test method.
--special-lookup
¶
Activates or disables the usage of our SPECIAL and extra rules whether possible.
Don’t want to use/apply the Special Rules - which are explained in the source column section?
This argument disables them all.
Default value: special: True
--special-lookup-only
¶
New in version 4.1.0b10.
Activates or disables the exclusive usage of the SPECIAL lookup as test method.
--whois-lookup
¶
New in version 4.0.0.
Activates or disables the usage of the WHOIS record (or better said the expiration date in it) when possible.
Don’t want to use or take the whois
date into consideration?
This argument allows you to disable it!
Default value: whois: True
Note
When you use the --syntax
no WHOIS data lookup will be performed
In other words: --syntax
overrules this argument
--whois-lookup-only
¶
New in version 4.1.0b10.
Activates or disables the exclusive usage of the WHOIS lookup as test method.
--reputation-lookup
¶
Todo
Check which of the reputation is alive or the code difference
Want to take the reputation data into consideration?
Activates or disables the usage of the reputation dataset when possible.
Default value: reputation: False
--reputation-lookup-only
¶
New in version 4.1.0b10.
Activates or disables the exclusive usage of the Reputation lookup as test method.
--syntax
¶
This code is to check the syntax of domains when the -f and URI’s when –url is used as source.
You should be able to use both -f
and code:-uf` at the same time with
--syntax
When you are using this flags there will not be performed any other test, such as the WHOIS or HTTP status code
Default value: syntax: False
Note
TIP: If you would like to gain some serious performance while testing
with --syntax
, We recommend you disable
–auto-continue
See note for --rpz
-t "seconds"
| --timeout "seconds"
¶
Sets the default timeout to apply to each lookup utilities every time it is possible to define a timeout.
Default value: timeout: 5
seconds
New in version 4.0.3b1.dev.
--max-http-retries "number"
¶
Sets the maximum number of retries for an HTTP request.
Default value: max_http_retries: 3
-ua "full string"
| --user-agent "full string"
¶
User defined user agent to use in the http status code lookup.
user_agent:
browser: chrome
platform: linux
Warning
If not given, we try to get the latest (automatically) for you
Example of how to change the default from CLI.
$ PyFunceble --user-agent "Mozilla/5.0 (X11; U; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36"
-vsc
| --verify-ssl-certificate
¶
Activates or disables the verification of the SSL/TLS certificate when testing for URL.
Default value: verify_ssl_certificate: False
Warning
If you activate the verification of the SSL/TLS certificate, you may get false-positive results.
Indeed if the certificate is not registered to the CA or is simply
invalid and the domain is still alive, you will always get
INACTIVE
as output.
DNS control¶
--dns
¶
By default, PyFunceble will use the system-wide DNS settings. This can be changed with the ability to configure which DNS-Servers you like PyFunceble to use during the test.
You set this up with the CLI command --dns
or insert it into your
personal .PyFunceble.yaml
You can add several separated by spaces and they will all be used in a order. (Kind of Round Robin style)
Default value: Follow OS DNS
==> server: null
$ PyFunceble --dns 127.0.1.53:5303 127.0.0.1 -f $DOMAIN_FILE
You can also set default DNS servers used for testing within the
my_project/.PyFunceble.yaml
file. (No secondary indent)
server:
- 1.2.3.4
- 5.6.7.8
- 9.10.11.12:5302
Warning
We expect a DNS server(s). If you add this flag but no DNS server(s) is
given. You’ll almost for certain get all results as INACTIVE
This could happen in case you use --dns -f
Note
You can specify the port number to be used on the DNS server if needed.
If you provide an IPv6, we are expecting the [ip]:port
format.
--dns-protocol
¶
Sets the protocol to use for the DNS queries.
Default value: protocol: UDP
Available values: UDP
, TCP
, HTTPS
, TLS
.
Case-Sensitive
$ PyFunceble --dns doh.powerdns.org --dns-protocol HTTPS -f $DOMAIN_FILE
$ PyFunceble --dns 192.0.2.2:53 --dns doh.powerdns.org --dns-protocol HTTPS
--follow-server-order
¶
New in version 4.0.0.
Let us follow or mix the order of usage of the given or found DNS server(s).
Default value: True
--trust-dns-server
¶
New in version 4.0.0.
Activates or disable the trust mode.
Default value: False
Note
When active, when the first read DNS server give us a negative response - without error - we take it as it it.
Otherwise, if not active, when the first read DNS server give us a negative response - without error - we still consolidate by checking all given/found server.
--dns-delay
¶
New in version 4.1.0b12.dev.
Sets the delay to apply between each DNS query.
Default value: 0.0
Note
When greater that 0.0
, a delay will be applied between each DNS
query.
Otherwise, if equal to 0.0, no delay will be applied.
Proxy control¶
--http-proxy
¶
New in version 4.1.0b12.dev.
By default, PyFunceble will use the environment-wide Proxy Settings. This can be changed with the ability to configure which Proxy Server you like PyFunceble to use during the test of subject.
You can set this up with the CLI command --http-proxy
or insert it
into your personal .PyFunceble.overwrite.yaml
Default value: Follow HTTP_PROXY variable
$ PyFunceble --http-proxy http://127.0.0.1:9899 -f $DOMAIN_FILE
You can also set default DNS servers used for testing within the
my_project/.PyFunceble.yaml
file. (No secondary indent)
global:
http: http://127.0.0.1:9899
Warning
We expect a Proxy Server. If you add this flag but no valid Proxy Server is
given. You’ll almost for certain get all results as INACTIVE
--https-proxy
¶
New in version 4.1.0b12.dev.
By default, PyFunceble will use the environment-wide Proxy Settings. This can be changed with the ability to configure which Proxy Server you like PyFunceble to use during the test of subject.
You can set this up with the CLI command --https-proxy
or insert it
into your personal .PyFunceble.overwrite.yaml
Default value: Follow HTTPS_PROXY variable
$ PyFunceble --https-proxy http://127.0.0.1:9899 -f $DOMAIN_FILE
You can also set default DNS servers used for testing within the
my_project/.PyFunceble.yaml
file. (No secondary indent)
global:
https: http://127.0.0.1:9899
Warning
We expect a Proxy Server. If you add this flag but no valid Proxy Server is
given. You’ll almost for certain get all results as INACTIVE
Databases¶
--inactive-database
¶
Switch the value of the usage of a database to store inactive domains of the currently tested list.
Default value: db_clean: 28
Day(s).
This argument will disable or enable the usage of a database which saves all
INACTIVE
and INVALID
domain of the given file over time.
--database-type
¶
Sets the database engine to use.
Default value: db_type: csv
Available values: csv
, mariadb
, mysql
, postgresql
.
--inactive-db
¶
Activates or disables the usage of a ‘database’ to store all ‘INACTIVE’ and ‘INVALID’ subject for continuous retest.
Configured value: inactive_db: True
-dbr "time"
| --days-between-db-retest "time"
¶
Sets the numbers of days since the introduction of a subject into the inactive dataset before it gets retested.
Default value: db_retest: 1
Day(s)
Note
This argument is only used if -db
or
inactive_database : true
(under .PyFunceble.yaml
) are
activated. See also –inactive-db
-wdb
| --whois-database
¶
Activates or disables the usage of a “database” to store the expiration date of all domains with a valid expiration date.
Default value: whois_db: True
Output control¶
-a
| --all
¶
Activates or disables the display of the all information in the table we print to stdout (screen).
Default value: all: False
Default:
Domain Status Source
----------------------------- ----------- ----------
pyfunceble.readthedocs.io ACTIVE SYNTAX
When :code:`all: True`:
Domain Status Expiration Date Source HTTP Code Checker
----------------------------- ----------- ----------------- ---------- ---------- -------------
pyfunceble.readthedocs.io ACTIVE Unknown NSLOOKUP 302 AVAILABILITY
--color
| --colour
¶
Activates or disables the coloration to STDOUT.
Default value: colour: True
Don’t want any colour ? This argument is for you!
--display-status
¶
New in version 4.0.0.
Sets the status that we are allowed to print to stdout.
Multiple space separated statuses can be given.
Default value: status: all
Available values: all
, ACTIVE
, INACTIVE
,
INVALID
, VALID
, SANE
, MALICIOUS
Default response
$ PyFunceble -d google-analytics.com duckduckgo.com --whois-lookup
Subject Status Source
---------------------------------------------------- ----------- ----------
duckduckgo.com ACTIVE DNSLOOKUP
google-analytics.com INACTIVE STDLOOKUP
Show only active and inactive
$ PyFunceble -d google-analytics.com duckduckgo.com --whois-lookup \
--display-status INACTIVE ACTIVE
Subject Status Source
---------------------------------------------------- ----------- ----------
duckduckgo.com ACTIVE DNSLOOKUP
google-analytics.com INACTIVE STDLOOKUP
Show only inactive
$ PyFunceble -d google-analytics.com duckduckgo.com --whois-lookup \
--display-status INACTIVE
Subject Status Source
---------------------------------------------------- ----------- ----------
google-analytics.com INACTIVE STDLOOKUP
Note
If you have provided more than one $DOMAIN_FILE as input source, then the printed status will be in same order as your $DOMAIN_FILE was given in the input.
For an example you can visit: github
-ex
| --execution
¶
Want to know the execution time of your test? Well, this argument will let you know!
Default value: execution_time: False
--hierarchical
¶
Activates or disables the sorting of the files content (output) in a hierarchical order.
Default value: hierarchical: False
This argument will output the result listed in a hierarchical order.
-h
| --hosts
¶
This argument will let the system know if it want to generate a hosts formatted result file for each status.
Default value: hosts: True
Note
There is an ongoing request to set the default value of hosts: False
You should be following this issue as it might affect your setup/results
later on.
Flip defaults for host
-ip "ip-address"
| --hosts-ip
“ip-address”¶
Sets the IP to prefix each lines of the hosts file.
Default value: 0.0.0.0
--logging-level
¶
New in version 4.0.0.
You can configure the logging level to be outputted in STDOUT (screen)
when you uses --no-files
. Default outputs to
output/__pyfunceble_loggging__/*.log
Please notice this is a separated folder at the same lever as your tested sources.
├── /tmp/pyfunceble/output/__pyfunceble_loggging__/
├── /tmp/pyfunceble/output/my-blacklist/
Optional values. (From less to more information)
|
--merge-output
¶
Activates or disables the merging of the outputs of all inputted files inside a single subdirectory as opposed to the normal behavior.
Default value: merge_output_dirs: False
--no-files
¶
Activates or disables the generation of any non-logs and status file(s).
Default value: no_file: False
See also
--output-location
¶
New in version 4.0.0.
This is used to direct the output location and matches PYFUNCEBLE_OUTPUT_LOCATION.
With this new option you no longer need to add the Global Variable but can append it directly to the CLI string.
$ PyFunceble --output-location /tmp/pyfunceble -f $DOMAIN_FILE
--unified-results
¶
Activates or disables the generation of the unified results file instead of the
divided output in individual subfolder under output/
.
Default value: unified_results: False
This argument disables the generation of the result.txt
file.
--percentage
¶
Activates or disables the display and generation of the percentage - file - of each status.
Default value: percentage: True
This argument will disable or enable the generation of the percentage of each status.
--registrar
¶
New in version 4.1.0b1.dev.
Activates or disables the display and generation of the (top) registrar stats - file.
Default value: registrar: False
--max-registrar
¶
New in version 4.1.0b1.dev.
Sets the maximum number of registrar to display.
Default value: max_registrar: 15
Note
This option does not take effect on the generated file.
--plain
¶
Activates or disables the generation of the generation of clean file(s).
This will output a file per status only containing the subject(s). (One record per line)
Default value: plain: True
See also
--dots
¶
CLI only: Activate or disables the display of dots or other characters when we skip the test of a subject.
CI only: If you combine the --ci --dots
we display a dot for each
record we tests.
Default value: dots: False
-q
| --quiet
¶
Activates or disables the display of output to the terminal.
Default value: quiet: False
--push-collection
¶
New in version 4.0.0.
Activates or disables the push of the test results into the collection API.
Default value: collection.push: False
Want to take submit data into the collection API ? This argument is for you.
Warning
This argument is useless if the PYFUNCEBLE_COLLECTION_API
environment
variable is not defined.
-s
| --simple
¶
Activates or disables the simple output mode.
Default value: simple: False
Want as less as possible data on screen? This argument returns as less as possible on screen!
Multiprocessing¶
-w
| --max-workers
¶
New in version 4.0.0.
Sets the number of maximal worker to use.
Keep in mind that the --max-workers
mostly - if not only - affects
the number of tester sub-processes. Because we want to safely write the
files, we still need a single process which read the submitted results and
generate the outputs.
The reason we added this to PyFunceble 4.0.0
is we don’t want to
have a wrongly formatted output file.
If you have more than 2 CPU cores/processes the default will be number of CPU - 2. Otherwise, it will 1.
Default value: max_workers: null
Note
If you have a CPU with 4 cores or Threads (depends on it’s age) Then the number of workers will be 4 - 2 = 2 workers
Warning
This section about max-workers is still under construction, but it is close to how it is working.
- This means you should be experimenting a bit your self.
To follow the “behind the scene” talk about the subject, please take a look at issue
CI / CD¶
--ci
¶
Activates or disables the Continuous Integration mechanism.
Default value: active: False
Note
If you combine this argument with the --quiet
argument, the test
will output a dotted line, where each dot (.
) represent one test
result or input which was skipped because it was previously tested.
Want to use PyFunceble under a supported CI infrastructure/network? This argument is suited for your needs!
--ci-max-minutes
¶
Sets the number of minutes to wait before starting to stop a CI session.
Default value: max_exec_minutes: 15
--ci-branch
¶
Sets our git working branch. This is the branch from where we are supposed to store the tests (excepts the final results).
Default value: branch: master
Note
Currently the branch need to exist, but there are being worked on a path
to have PyFunceble to create the sub-branch and finally merge it into the
--ci-distribution-branch
--ci-distribution-branch
¶
Sets our git distributions branch. This is the branch from where we are supposed to store and push the final results.
Default value: distribution_branch: master
Note
The difference between this and --ci-branch
is the fact
that this branch will get the (final) result only when the test is finished
under the given --ci-branch
.
As an example, this allows us to have 2 branches:
--ci-branch processing # (CI branch), for the tests with PyFunceble.
--ci-distribution-branch master # (CI distribution branch), for the
# distribution of the results of PyFunceble.
--ci-command "something"
¶
Changed in version 4.0.0.
Sets the command to execute before each commit (except the final one).
Default value: command: null
Note
In this example, something
should be a script or a program which
have to be executed when we reached the end of the given file.
Note
This argument is only used if --ci
or ci: true
(under
.PyFunceble.yaml
) are activated.
--ci-end-command "something"
¶
Changed in version 4.0.0.
Sets the command to execute before the final commit.
Default value: end_command: null
Note
In this example, something
should be a script or a program which
have to be executed when we reached the end of the given file.
Note
This argument is only used if --ci
or ci: true
(under
.PyFunceble.yaml
) are activated.
--ci-commit-message "message"
¶
Changed in version 4.0.0.
Sets the commit message to apply every time we have to apply a commit except for the really last one.
Default value: commit_message: "PyFunceble - AutoSave"
This argument allows us to set a custom commit message which is going to be used as a commit message when saving.
Note
This argument is only used if --ci
or ci: true
(under
.PyFunceble.yaml
) are used.
Note
This argument is only used if we have to split the work into multiple processes because a list is too long or the timeout is reached.
Warning
Please avoid the usage of [ci skip]
here.
--ci-end-commit-message
¶
Changed in version 4.0.0.
Sets the commit message to apply at the really end.
Default value: end_commit_message: "PyFunceble - Results"
Note
This argument is only used if --ci
or ci: true
(under
.PyFunceble.yaml
) are used.
Note
This argument is only used if we reached the end of the list we are or have to test.
-c
| --auto-continue
| --continue
¶
This argument is to used for auto-continuing from a previously under CI
Default value: autocontinue: False
This argument activates or deactivates the auto-continue subsystem. Indeed, as we can automatically continue if the script has been stopped, this switch allows us to disable or enable the usage of that specific subsystem.
--preload
¶
New in version 4.0.0.
Activates or disables the preloading of the input file(s) into the continue dataset before starting the tests.
The –preload argument - or its option counterpart - ping is given, we decode and load the given input files into the continue dataset before starting the test.
This reduces the waiting time while continuing a previous session.
Note
This argument is useless unless the auto continue subsystem is active.
The preloading may take some time depending of the size of the file to test, but this is the price for a smooth and better autocontinue. Especially under CI’s.
Global Variables¶
Here is the list of environment variables we use and how we use them if they are set.
Note
If used in a script like bash or a terminal directly you have to use the
export
as PyFunceble is running as sub-processes
Environment Variable | How to use them? |
PYFUNCEBLE_AUTO_CONFIGURATION |
Tell us if we have to install/update the configuration file automatically. |
PYFUNCEBLE_COLLECTION_API_TOKEN |
Sets the API token to use when pushing data into the collection API. |
PYFUNCEBLE_CONFIG_DIR |
Tell us the location of the directory to use as the configuration directory. |
PYFUNCEBLE_DB_CHARSET |
Tell us the MariaDB charset to use. |
PYFUNCEBLE_DB_HOST |
Tell us the host or the Unix socket (absolute file path) of the MariaDB database. |
PYFUNCEBLE_DB_NAME |
Tell us the name of the MariaDB database to use. |
PYFUNCEBLE_DB_PASSWORD |
Tell us the MariaDB user password to use. |
PYFUNCEBLE_DB_PORT |
Tell us the MariaDB connection port to use. |
PYFUNCEBLE_DB_USERNAME |
Tell us the MariaDB user-name to use. |
PYFUNCEBLE_DEBUG |
Tell us to log everything into the output/logs/*.log files. |
PYFUNCEBLE_DEBUG_LVL |
Sets the logging level to use. --logging-level |
PYFUNCEBLE_DEBUG_ON_SCREEN |
Tell us to log everything to stdout bool (true | false) |
PYFUNCEBLE_LOGGING_LVL |
Same as PYFUNCEBLE_DEBUG_LVL . --logging-level |
PYFUNCEBLE_OUTPUT_LOCATION |
Tell us where we should generate the output/ directory. |
APPDATA |
Used under Windows to construct/get the configuration directory if PYFUNCEBLE_CONFIG_DIR is not found. |
GH_TOKEN |
Tell us the GitHub token to set into the repository configuration when using PyFunceble under Travis CI. |
GL_TOKEN |
Tell us the GitLab token to set into the repository configuration when using PyFunceble under GitLab CI/CD. |
GIT_EMAIL |
Tell us the git.email configuration to set when using PyFunceble under any supported CI environment. |
GIT_NAME |
Tell us the git.name configuration to set when using PyFunceble under any supported CI environment. |
TRAVIS_BUILD_DIR |
Used to confirm that we are running under a Travis CI container. |
GITLAB_CI |
Used to confirm that we are running under a GitLab CI/CD environment. |
GITLAB_USER_ID |
Used to confirm that we are running under a GitLab CI/CD environment. |
Global overview¶
usage: pyfunceble [--show-completion {bash,zsh,tcsh}]
[-d DOMAINS [DOMAINS ...]] [-u URLS [URLS ...]]
[-f FILES [FILES ...]] [-uf URL_FILES [URL_FILES ...]]
[--adblock] [--cidr] [--complements] [--preload]
[--filter CLI_TESTING__FILE_FILTER] [--mining] [--rpz]
[--wildcard] [--chancy] [-c]
[--cooldown-time CLI_TESTING__COOLDOWN_TIME] [--local]
[--collection-preferred-origin {frequent,latest,recommended}]
[--collection-lookup] [--collection-lookup-only]
[--dns-lookup] [--dns-lookup-only] [--http] [--http-only]
[--netinfo-lookup] [--netinfo-lookup-only]
[--special-lookup] [--special-lookup-only] [--whois-lookup]
[--whois-lookup-only] [--reputation-lookup]
[--reputation-lookup-only] [--reputation] [--syntax]
[-t LOOKUP__TIMEOUT] [--max-http-retries MAX_HTTP_RETRIES]
[-ua USER_AGENT__CUSTOM] [-vsc]
[--dns DNS__SERVER [DNS__SERVER ...]]
[--dns-protocol {UDP,TCP,HTTPS,TLS}] [--follow-server-order]
[--trust-dns-server] [--dns-delay DNS__DELAY]
[--http-proxy PROXY__GLOBAL__HTTP]
[--https-proxy PROXY__GLOBAL__HTTPS] [--inactive-db]
[--database-type {csv,mariadb,mysql}]
[-dbr CLI_TESTING__DAYS_BETWEEN__DB_RETEST] [-wdb] [-a]
[-ex] [--colour]
[--display-status {all,ACTIVE,INACTIVE,VALID,INVALID,MALICIOUS,SANE} [{all,ACTIVE,INACTIVE,VALID,INVALID,MALICIOUS,SANE} ...]]
[--dots] [--hierarchical] [-h] [-ip CLI_TESTING__HOSTS_IP]
[--merge-output] [--no-files]
[--output-location OUTPUT_LOCATION] [--unified-results]
[--percentage] [--registrar]
[--max-registrar CLI_TESTING__DISPLAY_MODE__MAX_REGISTRAR]
[--plain] [-q] [--push-collection] [-s]
[-w CLI_TESTING__MAX_WORKERS]
[--ci-max-minutes CLI_TESTING__CI__MAX_EXEC_MINUTES] [--ci]
[--ci-branch CLI_TESTING__CI__BRANCH]
[--ci-distribution-branch CLI_TESTING__CI__DISTRIBUTION_BRANCH]
[--ci-command CLI_TESTING__CI__COMMAND]
[--ci-end-command CLI_TESTING__CI__END_COMMAND]
[--ci-commit-message CLI_TESTING__CI__COMMIT_MESSAGE]
[--ci-end-commit-message CLI_TESTING__CI__END_COMMIT_MESSAGE]
[--help] [-v]
PyFunceble - The tool to check the availability or syntax of domain, IP or URL.
options:
--show-completion {bash,zsh,tcsh}
Show Shell completion script and exit.
--help Show this help message and exit.
-v, --version Show the version of PyFunceble and exit.
Test sources:
-d DOMAINS [DOMAINS ...], --domain DOMAINS [DOMAINS ...]
Test one or more domains, separated by spaces.
When this option is used, no output files are generated.
-u URLS [URLS ...], --url URLS [URLS ...]
Test one or more full URL, separated by spaces.
-f FILES [FILES ...], --file FILES [FILES ...]
Read a local or remote (RAW link) file and test all domains inside it.
If remote (RAW link) file is given, PyFunceble will download it,
and test the content of the given RAW link as if it was a locally stored file.
-uf URL_FILES [URL_FILES ...], --url-file URL_FILES [URL_FILES ...]
Read a local or remote (RAW link) file and test all (full) URLs inside it.
If remote (RAW link) file is given, PyFunceble will download it,
and test the content of the given RAW link as if it was a locally stored file.
This argument test if an URL is available. It ONLY test full URLs.
Source filtering, decoding, conversion and expansion:
--adblock Activates or deactivates the decoding of the adblock format.
Configured value: False
--cidr Activates or disables the expansion of CIDR formatted
addresses.
Configured value: False
--complements Activates or disables the generation and test of the
complements.
A complement is for example `example.org` if 'www.example.org'
is given and vice-versa.
Configured value: False
--preload Activates or disables the preloading of the input
file(s) into the continue dataset before starting the tests.
This reduces the waiting time while continuing a previous
session.
Note: This is useless when the auto continue subsystem is not active.
Configured value: False
--filter CLI_TESTING__FILE_FILTER
Regex to match in order to test a given line.
Configured value: None
--mining Activates or disables the mining subsystem.
Configured value: False
--rpz Activates or disables the decoding of RPZ policies
from each given input files.
Configured value: False
--wildcard Activates or disables the decoding of wildcards for
each given input files.
Configured value: False
Test control:
--chancy, --ludicrous
Activates a chancy mode that unleashes the safety
workflow in place.
WARNING: You shouldn't have to use this unless you feel really lucky
and trust your machine. This mode makes things look 'fast',
but it may produce some unexpected results if N process
simultaneously write the same output file.
This mode makes the graphical CLI output unparsable - either.
MAY THE FORCE BE WITH YOU!
Configured value: False
-c, --auto-continue, --continue
Activates or disables the autocontinue subsystem.
Configured value: False
--cooldown-time CLI_TESTING__COOLDOWN_TIME
Sets the cooldown time (in second) to apply between
each test.
Configured value: 0.0
--local Activates or disables the consideration of the test(s)
in or for a local or private network context.
Configured value: False
--collection-preferred-origin {frequent,latest,recommended}
Sets the preferred status origin.
Configured value: 'recommended'
--collection-lookup Activates or disables the usage of the Collection lookup
whether possible.
Configured value: False
--collection-lookup-only
Only perform a Collection lookup.
--dns-lookup Activates or disables the usage of the DNS lookup
whether possible.
Configured value: True
--dns-lookup-only Only perform a DNS lookup.
--http, --http-status-code-lookup
Switch the value of the usage of HTTP code.
Configured value: True
--http-only, --http-status-code-lookup-only
Only perform a HTTP Code lookup.
--netinfo-lookup Activates or disables the usage of the network
information (or network socket) whether possible.
Configured value: True
--netinfo-lookup-only
Only perform a network information (or networket socket) lookup.
--special-lookup Activates or disables the usage of our SPECIAL and
extra rules whether possible.
Configured value: True
--special-lookup-only
Only perform a SPECIAL lookup.
--whois-lookup Activates or disables the usage of the WHOIS record
(or better said the expiration date in it) whether possible.
Configured value: True
--whois-lookup-only Only perform a WHOIS lookup.
--reputation-lookup Activates or disables the usage of the reputation
dataset whether possible.
Configured value: False
--reputation-lookup-only
Only perform a reputation lookup.
--reputation Activates or disables the reputation checker.
Configured value: False
--syntax Activates or disables the syntax checker.
Configured value: False
-t LOOKUP__TIMEOUT, --timeout LOOKUP__TIMEOUT
Sets the default timeout to apply to each lookup
utilities every time it is possible to define a timeout.
Configured value: 5
--max-http-retries MAX_HTTP_RETRIES
Sets the maximum number of retries for an HTTP request.
Configured value: 0
-ua USER_AGENT__CUSTOM, --user-agent USER_AGENT__CUSTOM
Sets the user agent to use.
If not given, we try to get the latest (automatically) for you.
-vsc, --verify-ssl-certificate
Activates or disables the verification of the SSL/TLS
certificate when testing for URL.
Configured value: False
DNS control:
--dns DNS__SERVER [DNS__SERVER ...]
Sets one or more (space separated) DNS server(s) to use during testing.
To specify a port number for the DNS server you append
it as :port [ip:port].
If no port is specified, the default DNS port (53) is used.
Configured value: <BoxList: ['1.1.1.1']>
--dns-protocol {UDP,TCP,HTTPS,TLS}
Sets the protocol to use for the DNS queries.
Configured value: 'UDP'
--follow-server-order
Let us follow or mix the order of usage of the given
or found DNS server(s).
Configured value: True
--trust-dns-server Activates or disable the trust mode.
When active, when the first read DNS server give us a negative
response - without error - we take it as it it.
Otherwise, if not active, when the first read DNS server give us
a negative response - without error - we still consolidate by
checking all given/found server.
Configured value: False
--dns-delay DNS__DELAY
Sets the delay (in seconds) to apply between each DNS
queries.
Configured value: 0.0
Proxy control:
--http-proxy PROXY__GLOBAL__HTTP
Sets the proxy to use when testing subjects over HTTP.
Configured value: None
--https-proxy PROXY__GLOBAL__HTTPS
Sets the proxy to use when testing subjects over HTTPS.
Configured value: None
Databases:
--inactive-db Activates or disables the usage of a 'database' to
store all 'INACTIVE' and 'INVALID' subject for continuous retest.
Configured value: False
--database-type {csv,mariadb,mysql}
Sets the database engine to use.
You can choose between the following: `csv | mariadb | mysql`
Configured value: 'csv'
-dbr CLI_TESTING__DAYS_BETWEEN__DB_RETEST, --days-between-db-retest CLI_TESTING__DAYS_BETWEEN__DB_RETEST
Sets the numbers of days since the introduction of
subject into the inactive dataset before it gets retested.
Configured value: 1
-wdb, --whois-database
Activates or disables the usage of a 'database' to
store the expiration date of all domains with a valid
expiration date.
Configured value: True
Output control:
-a, --all Activates or disables the display of the all
information in the table we print to stdout.
Configured value: False
-ex, --execution Activates or disables the display of the execution time.
Configured value: False
--colour, --color Activates or disables the coloration to STDOUT.
Configured value: True
--display-status {all,ACTIVE,INACTIVE,VALID,INVALID,MALICIOUS,SANE} [{all,ACTIVE,INACTIVE,VALID,INVALID,MALICIOUS,SANE} ...]
Sets the status that we are allowed to print to STDOUT.
Multiple space separated statuses can be given.
Configured value: 'all'
--dots Activate or disables the display of dots or other
characters when we skip the test of a subject.
Configured value: False
--hierarchical Activates or disables the sorting of the files
content (output) in a hierarchical order.
Configured value: False
-h, --host Activates or disables the generation of the
hosts file(s).
Configured value: False
-ip CLI_TESTING__HOSTS_IP, --hosts-ip CLI_TESTING__HOSTS_IP
Sets the IP to prefix each lines of the hosts file.
Configured value: '0.0.0.0'
--merge-output Activates or disables the merging of the outputs of all
inputted files inside a single subdirectory as opposed to the
normal behavior.
Configured value: False
--no-files Activates or disables the generation of any non-logs
file(s).
Configured value: False
--output-location OUTPUT_LOCATION
Sets the location where we are supposed to generation
the output directory from.
Configured value: '/home/pyfunceble-dev/repositories/github/source/PyFunceble'
--unified-results Activates or disables the generation of the unified
results file instead of the divided ones.
Configured value: False
--percentage Activates or disables the display and generation
of the percentage - file - of each status.
Configured value: True
--registrar Activates or disables the display and generation
of the registrar - file - status at the end of a test.
The registrar file contains the top domain registrar found
while testing.
Configured value: True
--max-registrar CLI_TESTING__DISPLAY_MODE__MAX_REGISTRAR
Sets the maximal number of registrar to display.
Note: This argument has no effect when the --registrar
argument is not set. This argument only takes effect on
display but not
in the log file
Configured value: 15
--plain Activates or disables the generation of the
RAW file(s). What is meant is a list with only a list of
subject (one per line).
Configured value: True
-q, --quiet Activates or disables the display of output to the
terminal.
Configured value: False
--push-collection Activates or disables the push of test result into the
collection API.
Configured value: False
-s, --simple Activates or disables the simple output mode.
Configured value: False
Multiprocessing:
-w CLI_TESTING__MAX_WORKERS, --max-workers CLI_TESTING__MAX_WORKERS
Sets the number of maximal workers to use.
If not given, 20 (based on the current machine) will be applied.
Configured value: None
CI / CD:
--ci-max-minutes CLI_TESTING__CI__MAX_EXEC_MINUTES
Sets the number of minutes to wait before starting
to stop a CI session.
Configured value: 15
--ci Activates or disables the Continuous Integration
mechanism.
Configured value: False
--ci-branch CLI_TESTING__CI__BRANCH
Sets our git working branch. This is the branch
from where we are supposed to store the tests
(excepts the final results).
Configured value: 'master'
--ci-distribution-branch CLI_TESTING__CI__DISTRIBUTION_BRANCH
Sets our git distributions branch. This is the
branch from where we are supposed to store and push
the final results.
Configured value: 'master'
--ci-command CLI_TESTING__CI__COMMAND
Sets the command to execute before each commit
(except the final one).
Configured value: None
--ci-end-command CLI_TESTING__CI__END_COMMAND
Sets the command to execute before the final commit.
Configured value: None
--ci-commit-message CLI_TESTING__CI__COMMIT_MESSAGE
Sets the commit message to apply every time we have
to apply a commit except for the really last one.
Configured value: 'PyFunceble - AutoSave'
--ci-end-commit-message CLI_TESTING__CI__END_COMMIT_MESSAGE
Sets the commit message to apply at the really end.
Configured value: 'PyFunceble - Results'
For an in-depth usage, explanation and examples of the arguments,
you should read the documentation at https://pyfunceble.readthedocs.io/en/latest/
Crafted with ♥ by Nissar Chababy (@funilrys) with the help of
https://pyfunceble.github.io/#/contributors && https://pyfunceble.github.io/#/special-thanks
Github Actions CI/CD¶
Example of how you can run PyFunceble inside Github Actions.
name: PyFunceble CI tests
on:
push:
branches:
- "master"
pull_request:
branches:
- "master"
schedule:
- cron: "0 * * * *"
env:
PYFUNCEBLE_AUTO_CONFIGURATION: "YES"
GIT_NAME: "${{ secrets.GIT_BOT_NAME }}"
GIT_EMAIL: "${{ secrets.GIT_BOT_EMAIL }}"
PYFUNCEBLE_CONFIG_DIR: "${{ github.workspace }}/.pyfunceble"
GITHUB_TOKEN: "${{ secrets.BOT_REPO_PAT }}"
jobs:
single:
name: Run PyFunceble with a single domain
runs-on: "${{ matrix.os }}"
strategy:
fail-fast: false
matrix:
python_version:
- "3.9.1"
os:
- ubuntu-latest
steps:
- uses: actions/checkout@v3
name: Clone repository
with:
token: "${{ secrets.BOT_REPO_PAT }}"
- name: Set up Python ${{ matrix.python_version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python_version }}
- name: Install dependencies
run: |
pip install --pre PyFunceble-dev
- name: Get PyFunceble version
run: |
PyFunceble --version
- name: Run PyFunceble
run: |
PyFunceble -a --logging-level critical -d github.com
file_and_push:
name: Run PyFunceble against a file and push result to repository
runs-on: "${{ matrix.os }}"
strategy:
fail-fast: false
matrix:
python_version:
- "3.9.1"
os:
- ubuntu-latest
steps:
- uses: actions/checkout@v3
name: Clone repository
with:
token: "${{ secrets.BOT_REPO_PAT }}"
- name: Set up Python ${{ matrix.python_version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python_version }}
- name: Install dependencies
run: |
pip install --pre PyFunceble-dev
- name: Get PyFunceble version
run: |
PyFunceble --version
- name: Run PyFunceble
run: |
PyFunceble -a --ci --logging-level critical -f test.list
GitLab Runner CI/CD¶
As we offer an argument named --ci
which will
autosave in a GitLab CI/CI environment, this document try to
describe hot it works!
Configuration¶
Personal Access Token¶
A personal access token is needed in order for PyFunceble to automatically push the results.
You should get a personal GitLab access token with
the read_repository
and write_repository
scopes.
Once created and copied in a safe place, create a new masked variable
named GL_TOKEN
inside the CI/CD settings of your project.
The value of the variable should be the newly generated personal
access token.
.gitlab-ci.yml
¶
Note
This part only present a commented .gitlab-ci.yml
.
This is just an example do not take the following as
necessarly true.
You’re invited to submit changes if something stated in this document is wrong.
# Python needed, so we use the python image.
image: python:latest
variables:
# This is the Git name we have to set. (git config user.name)
GIT_EMAIL: "dead-hosts@funilrys.com"
# This is the Git Email we have to set. (git config user.email)
GIT_NAME: "GitLab CI/CD"
before_script:
# We install the development version of PyFunceble.
# If you prefer the stable version replace `pyfunceble-dev`
# with `pyfunceble`.
- pip3 install PyFunceble-dev
run:
script:
# Let's say we want our results and our PyFunceble
# infrastructure to be saved in a directory called `PyFunceble-tests`
# We move inside it.
- cd PyFunceble-tests
# We test the file `my_awesome_list` which is located inside the current directory.
# Note: we precise the `--ci` argument here,
# but you work without it if you set `ci: true` inside your `.PyFunceble.yaml`
- PyFunceble --ci -f my_awesome_list --plain
Travis container CI/CD¶
As we offer an argument named --ci
to activate
the usage of PyFunceble in a Travis CI instance,
we document here what you need to know!
Configuration¶
Note
This part only present a commented .travis.yml
so that you can understand where to start.
If you need more practical examples, feel free to report to one of Dead-Hosts repositories which use PyFunceble with Travis CI.
env:
global:
# The following is your encrypted GitHub API key.
# Indeed as we are going to push to the repository, this is needed.
#- GH_TOKEN: # This can be set in the travis-ci https://travis-ci.com/repo/settings as 'Environment Variables'
# or as below: secure: encrypted code
- secure: QQdKFquFFojFT9XJ1XZp4EMoDTVoXFgqZq8XU+sCVf+pJQR6d/oKBp8rnSTCnZizWOQXUjGXUUxUpSG/dYGyBLjo3rH3rsn9ciZHVfubxbwK860w4sqibl4DvhCv2rdsFtvzXnhm4P9OL3i+krKdewh9fxpNyUU58qOgfnS7mK9FcFhb8z5ak2sxU2XRZedwm6Ro0oyVKs8kFkL4YaADfNyAHlGTfr9rVmE52WXQXQENktb9gFgR2A8ZnmLy0BCMZGkPDShJnjRDWD4DErtasLmLQvWpzOBwdbVJTY6U9KDRXVNdC9lp5E5Ba/dc0y36q6vjfgJR+QchetOtHgNbKYbLB8c26Di90OZCFJsxMNcl1Wct4qFPXkFGvjXrISW6pbdPL5Plto0Ig3iLiulhYOPVArysMIk9ymtSXP+WE7VWX01LQ1fEkIoSfeVZ2caTnCmTsoHVGRRe978CojKaT7yU45kb15hcyDrzptQ8EP2hfxeh5F7KtueQ6Rsb9LFDZMkMDKflZn6a+bRhESlmWWmYB9stzGzTurQA1E1bcSACJ8A8hG5nHBzZYJ2S+OY0PE7UdyOJ0JK0qe/67d+F9ocQdIoFpDDTdgIjHerQnD2wRg1aKPzLDb4jJTpqgr5ssPrqUAKl3st7gyaAZzCEADPDnIBDjOJS+mFWbx9DKgc=
# This is the Git name we have to set. (git config user.name)
- GIT_NAME: Travis CI
# This is the Git Email we have to set. (git config user.email)
- GIT_EMAIL: dead-hosts@funilrys.com
# This is the language we use.
language: python
# This is the python version we are going to use for the tests.
# Note: you can add any 3.x version to the list.
python:
- "3.8"
# The following will tell Travis CI to ends as fast as possible.
matrix:
fast_finish: true
# Here we are setting what Travis CI have to cache.
cache:
# We are caching pip3 as we use it to install PyFunceble
- pip3
install:
# We install the development version of PyFunceble. If you prefer the stable version replace
# `pyfunceble-dev` with `pyfunceble`.
- pip3 install pyfunceble-dev
# Our tests start here.
script:
# Let's say we want our results and our PyFunceble infrastructure to be saved in a directory
# called `PyFunceble-tests`
# We move inside it.
- cd PyFunceble-tests
# We test the file `my_awesome_list` which is located inside the current directory.
# Note: we precise the `--ci` argument here,
# but you work without it if you set `travis: true` inside your `.PyFunceble.yaml`
- PyFunceble --ci -f my_awesome_list --plain
# The following initiate email notification logic.
notifications:
# As we want to get a mail on failure and on status change, we set the following.
on_success: change
on_failure: always
Getting a GitHub token¶
For the secure
index of the .travis.yml
file, you have to generate a new GitHub token.
After you got your token, please write it or save it in a safe place as you’re going to need it every time you’re going to interact with Travis CI.
Note
The scope to set is public_repo
but you can also set others depending on your needs.
Encrypting the token for future usage under the Travis CIs’ containers¶
To encrypt the token simply replace and execute the following according to your personal case.
$ travis encrypt 'GH_TOKEN=theGeneratedToken' -r 'The content of TRAVIS_REPO_SLUG' --add
Warning
Please do not execute the following explicitly without replacing theGeneratedToken
with your previously generated GitHub token and The content of TRAVIS_REPO_SLUG
with
your repository slug.
Note
The usage of --add
ensure that the travis
program automatically add the secure
index to the .travis.yml
file.
Deprecated arguments¶
The following arguments have previous been in use by PyFunceble, these are now deprecated and should in some cases be replaced with alternative args.
In other cases the old feature have either been included into a other functional or simply been removed as it was obselete to keep the function within Pyfunceble v4.x.
This is happening because PyFunceble is evolving over time and features comes and goes.
-nw
| --no-whois
¶
Deprecated since version 4.0.0.
Replacement: –whois-lookup
--shadow-file
| --shadow
¶
Deprecated since version 4.0.0.
Replacement: N/A
--use-reputation-data
¶
Deprecated since version 4.0.0.
Replacement: reputation-lookup
--dns-lookup-over-tcp
¶
Deprecated since version 4.0.0.
See: –dns-protocol
-db | --database
¶
Deprecated since version 4.0.0.
Replacement: –inactive-db
-dbc "something" | --days-between-db-clean
¶
Deprecated since version 4.0.0.
Replacement: N/A
-json
¶
Deprecated since version 4.0.0.
Replacement: N/A
-nu | --no-unified
¶
Deprecated since version 4.0.0.
Replacement: –unified-results
-ns|--no-special
Deprecated since version 4.0.0.
Replacement: –special-lookup
In the .code:.PyFunceble_production.yaml the value have changed from
no_special
to special
--split
¶
Deprecated since version 4.0.0.
Replacement: –unified-results
--store-whois
¶
Deprecated since version 4.0.0.
Replacement: N/A
-m | --multiprocess
¶
Deprecated since version 4.0.0.
Integrated into -w | –max-workers
-p | --processes
¶
Deprecated since version 4.0.0.
Replacement: -w | –max-workers
--multiprocess-merging-mode
¶
Deprecated since version 4.0.0.
Replacement: N/A
--autosave-minutes
¶
Deprecated since version 4.0.0.
Replacement: –ci-max-minutes
--cmd
¶
Deprecated since version 4.0.0.
Replacement: –ci-command