PyFunceble.cli.threads package¶

Submodules¶

PyFunceble.cli.threads.base module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the base of our threads classes.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.base.ThreadsBase(output_queue: Union[queue.Queue, Tuple[queue.Queue], None] = None)[source]¶

Bases: object

Provides the base of all classes.

_output_queue = None¶

add_to_output_queue(data: Any) → PyFunceble.cli.threads.base.ThreadsBase[source]¶

Adds the given data to the output queue.

Parameters:	data – The data to add into the output queue.

add_to_the_queue(data: Any) → PyFunceble.cli.threads.base.ThreadsBase[source]¶

Adds the given data to the current queue.

Parameters:	data – The data to add into the queue.

ensure_output_queue_is_given()[source]¶

Ensures that the output queue was given before launching the decorated method.

Raises:	RuntimeError – When the thread was not launched.

ensure_thread_is_initiated()[source]¶

Ensures that the thread was initiated before launching the decorated method.

Raises:	RuntimeError – When the thread was not launched.

ensure_thread_is_not_initiated()[source]¶

Ensures that the thread was not initiated before launching the decorated method.

Raises:	RuntimeError – When the thread was already launched.

is_failed() → bool[source]¶: Checks if the current thread has failed.

is_initiated() → bool[source]¶: Checks if the current thread was initiated.

is_running() → bool[source]¶: Checks if the current thread is running.

output_queue¶: Provides the current state of the _output_queue attribute.

send_stop_signal() → PyFunceble.cli.threads.base.ThreadsBase[source]¶: Puts the stop message into the queue.

set_output_queue(value: Union[queue.Queue, Tuple[queue.Queue]])[source]¶

Sets the output queue with the given queue.

Parameters:	value – The value to set.

static should_be_ignored(subject: str) → bool[source]¶: Checks if the given subject should be ignored.

start(*, daemon: bool = True) → PyFunceble.cli.threads.base.ThreadsBase[source]¶

Starts the threads itself.

Parameters:	daemon – Tells us to starts the threads as a daemon.

target() → None[source]¶: The target to give to the thread. This is where the magic happens :-)

the_queue = None¶

the_thread = None¶

thread_name = 'pyfunceble'¶

wait() → PyFunceble.cli.threads.base.ThreadsBase[source]¶: Stops the threads itself.

PyFunceble.cli.threads.continue_producer module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the logic behind the threads which is supposed to handle everything related to the autocontinue.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.continue_producer.ContinueProducerThread(output_queue: Optional[queue.Queue] = None)[source]¶

Bases: PyFunceble.cli.threads.producer_base.ProducerThreadBase

Provides our autocontienue handler logic.

The thread behind this object, will read the_queue, and produce into the autocontinue dataset.

dataset = None¶

run_continue(test_dataset: dict, test_result: PyFunceble.checker.status_base.CheckerStatusBase) → None[source]¶

Reads the given test dataset, interpret it and add the given test result to the dataset if needed.

Parameters:	test_dataset – The test dataset as per protocol. test_result_dict – The test result object.

target() → None[source]¶: int This is our core logic. Everything starts here!

thread_name = 'pyfunceble_continue_producer'¶

PyFunceble.cli.threads.file_producer module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the logic behind the threads which is supposed to generate outputs to the files or other formats.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.file_producer.FileProducerThread(output_queue: Optional[queue.Queue] = None)[source]¶

Bases: PyFunceble.cli.threads.producer_base.ProducerThreadBase

Provides our file producer thread logic.

The thread behind this object, will read the_queue, and procude the outputs to the files or other storage formats.

counter = None¶

file_printer = None¶

run_counter(test_dataset: dict, test_result: PyFunceble.checker.status_base.CheckerStatusBase) → None[source]¶

Runs the counter of the current file.

Parameters:	test_dataset – The test dataset as per protocol. test_result_dict – The test result object.

run_file_printer(test_dataset: dict, test_result: PyFunceble.checker.status_base.CheckerStatusBase) → None[source]¶

Runs the file printer if necessary.

Parameters:	test_dataset – The test dataset as per protocol. test_result_dict – The test result object.

run_ignored_file_printer(test_dataset: dict, test_result: str) → None[source]¶: Runs the file printer assuming that the given test dataset has been ignored by the test runner.

status_file_generator = None¶

target() → None[source]¶: int This is our core logic. Everything starts here!

thread_name = 'pyfunceble_file_producer'¶

PyFunceble.cli.threads.file_sorter module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the logic behind the threads which is supposed to sort the content of all interesting files.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.file_sorter.FileSorterThread(output_queue: Union[queue.Queue, Tuple[queue.Queue], None] = None)[source]¶

Bases: PyFunceble.cli.threads.base.ThreadsBase

Provides our file sorter thread logic. The main idea is that we start read the given protocol and sort all the files that needs to be sorted.

MAX_LINES = 1000¶

static get_files_to_sort_content(output_dir: str) → List[str][source]¶: Provides the list of files to work with.

process_sorting(file: str) → None[source]¶

Process the sorting of the given file.

The idea is to split the file piece by piece and at the end join all sorted files. For that job, we create a temporary directory which will store the temporary files.

target() → None[source]¶: This is our core logic. Everything starts here!

thread_name = 'pyfunceble_file_sorter'¶

PyFunceble.cli.threads.inactive_producer module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the logic behind the threads which is supposed to handle everything related to the inactive dataset.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.inactive_producer.InactiveProducerThread(output_queue: Optional[queue.Queue] = None)[source]¶

Bases: PyFunceble.cli.threads.producer_base.ProducerThreadBase

Provides our inactive handler logic.

The thread behind this object, will read the_queue, and produce into the inactive dataset.

dataset = None¶

run_inactive(test_dataset: dict, test_result: PyFunceble.checker.status_base.CheckerStatusBase) → None[source]¶

Processes the update of a dataset which was previously inactive.

The idea is that if the status is OK, we just remove it from the database. Otherwise, we just keep it in there :-)

target() → None[source]¶: int This is our core logic. Everything starts here!

thread_name = 'pyfunceble_inactive_producer'¶

PyFunceble.cli.threads.migrator module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the logic behind the threads which is supposed to run all our migrators.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.migrator.MigratorThread(output_queue: Union[queue.Queue, Tuple[queue.Queue], None] = None)[source]¶

Bases: PyFunceble.cli.threads.base.ThreadsBase

Provides our migrator thread logic. The main idea is that we start all our migrator inside multiple threads and wait for them to run.

continuous_integration = None¶

static done_callback(func: concurrent.futures._base.Future) → None[source]¶

This method will be executed after each task run.

Raises:	Exception – The the task has some exception.

hashes_file_cleanup_target() → None[source]¶: Provides the target for the cleanup of the hashes file.

json2csv_inactive_target() → None[source]¶: Provides the target for the inactive database migrator.

json2csv_whois_target() → None[source]¶: Provides the target for the whois database migrator.

mariadb_file_and_status_target() → None[source]¶: Provides the target for the migration of the pyfunceble_file and pyfunceble_status tables.

mariadb_whois_record_idna_subject_target() → None[source]¶: Provides the target for the whois addition of the missing idna_subject column.

mining_file_cleanup_target() → None[source]¶: Provides the target for the cleanup of the mining file.

target() → None[source]¶: This is our new target.

thread_name = 'pyfunceble_migrator'¶

PyFunceble.cli.threads.miner module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the logic behind the threads which is supposed to mine the dataset to later test.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.miner.MinerThread(output_queue: Optional[queue.Queue] = None)[source]¶

Bases: PyFunceble.cli.threads.base.ThreadsBase

Provides our miner thread logic. The main idea is that we read our queue, mine (or at least look for it) and write the new subject to test into the output queue.

The thread behind this object, will read the_queue, and write the mined subject into the output_queue attribute.

continuous_integration = None¶

static mine_from(subject: str) → Optional[List[str]][source]¶

Given the subject to work from, try to get the related subjects.

Parameters:	subject – The URL to start from.

target() → None[source]¶: This is our core logic. Everything starts here!

thread_name = 'pyfunceble_mining'¶

PyFunceble.cli.threads.producer_base module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the base of all our producer threads.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.producer_base.ProducerThreadBase(output_queue: Union[queue.Queue, Tuple[queue.Queue], None] = None)[source]¶

Bases: PyFunceble.cli.threads.base.ThreadsBase

Provides the base of all our producer threads.

The thread behind this object, will read the_queue, and procude the outputs.

INACTIVE_STATUSES = ('INACTIVE', 'INVALID')¶

should_we_block_printer(test_dataset: dict, test_result: PyFunceble.checker.status_base.CheckerStatusBase) → bool[source]¶

Checks if we should block the printer.

The reason behindn this is that we don’t want to generate an output when a subject was already into the inactive database.

Parameters:	test_dataset – The test dataset. test_result – The test result

static should_we_ignore(test_result: PyFunceble.checker.status_base.CheckerStatusBase) → bool[source]¶

Checks if the we should ignore the given datasets.

Parameters:	test_dataset – The test dataset. test_result – The test result

thread_name = 'pyfunceble_producer'¶

PyFunceble.cli.threads.stdout_producer module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the logic behind the threads which is supposed to generate outputs to the STDOUT.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.stdout_producer.StdoutProducerThread(output_queue: Optional[queue.Queue] = None)[source]¶

Bases: PyFunceble.cli.threads.producer_base.ProducerThreadBase

Provides our STDOUT producer thread logic.

The thread behind this object, will read the_queue, and procude the outputs to stdout.

header_already_printed = False¶

run_stdout_printer(test_result: PyFunceble.checker.status_base.CheckerStatusBase) → None[source]¶

Runs the stdout printer if necessary.

Parameters:	test_dataset – The test dataset. test_result – The test result object.

send_for_mining(test_dataset: dict, test_result: PyFunceble.checker.status_base.CheckerStatusBase) → None[source]¶: Sends the test dataset to the mining queue (if given).

static shoud_we_print_status(status: str) → bool[source]¶

Checks if we are allowed to print based on the given status.

Parameters:	status – The status to check.

stdout_printer = None¶

target() → None[source]¶: int This is our core logic. Everything starts here!

thread_name = 'pyfunceble_stdout_producer'¶

PyFunceble.cli.threads.tester module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the logic behind the threads which is supposed to test the given subject.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.tester.TesterThread(output_queue: Union[queue.Queue, Tuple[queue.Queue], None] = None)[source]¶

Bases: PyFunceble.cli.threads.base.ThreadsBase

Provides our tester thread logic.

The thread behind this object, will read the_queue, test all entries until a stop is given and store the result into output_queue.

continuous_integration = None¶

static done_callback(func: concurrent.futures._base.Future) → None[source]¶

This method will be executed after each task run.

Raises:	Exception – The the task has some exception.

get_status(test_dataset: dict) → Optional[PyFunceble.checker.status_base.CheckerStatusBase][source]¶

This is our part our our target. Meaning that you shouldn’t be playing with this outside the CLI.

This method reads the given dataset, starts the test and return the result of it after putting it into the declared output queue.

Please note a slighly side effect of this method. Indeed, None will be returned if the subject to test should be ignored.

static get_testing_object(subject_type: str, checker_type: str) → PyFunceble.checker.base.CheckerBase[source]¶

Provides the object to use for testing.

Raises:	ValueError – When the given subject type is unkown.

target() → None[source]¶: This is our core logic. Everything starts here!

thread_name = 'pyfunceble_tester'¶

PyFunceble.cli.threads.threading module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides our own version of some of object provided by the standard threading library.

Note

Our threads submodules or class does not extends the threading module. They are jus there to clarify our workflow for future contributors :-)

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.threading.Thread(group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None)[source]¶

Bases: threading.Thread

Overwrites and adds some functionalities to the threading.Thread class.

exception = None¶

is_failed() → bool[source]¶: Checks if the current thread has failed.

run() → None[source]¶: Runs the thread (normally), but catch the exception for our future purpose.

PyFunceble.cli.threads.utils module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides some utilities.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

PyFunceble.cli.threads.utils.wait_until_completion(tasks: List[concurrent.futures._base.Future], *, raise_exc: bool = False) → None[source]¶

Wait untill all tasks are done.

Parameters:	tasks – The list of submitted tasks (so far). raise_exc – Allows the raising of the exception (if exists) of the first failed task.

PyFunceble.cli.threads.whois_producer module¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides the logic behind the threads which is supposed to handle everything related to the WHOIS dataset.

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

class PyFunceble.cli.threads.whois_producer.WhoisProducerThread(output_queue: Optional[queue.Queue] = None)[source]¶

Bases: PyFunceble.cli.threads.producer_base.ProducerThreadBase

Provides our WHOIS dataset handler logic.

The thread behind this object, will read the_queue, and produce into the whois dataset.

dataset = None¶

run_whois_backup(test_result: PyFunceble.checker.status_base.CheckerStatusBase) → None[source]¶

Runs the WHOIS record backup.

Parameters:	test_result – The test result object.

target() → None[source]¶: int This is our core logic. Everything starts here!

thread_name = 'pyfunceble_whois_producer'¶

Module contents¶

The tool to check the availability or syntax of domain, IP or URL.

██████╗ ██╗   ██╗███████╗██╗   ██╗███╗   ██╗ ██████╗███████╗██████╗ ██╗     ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║   ██║████╗  ██║██╔════╝██╔════╝██╔══██╗██║     ██╔════╝
██████╔╝ ╚████╔╝ █████╗  ██║   ██║██╔██╗ ██║██║     █████╗  ██████╔╝██║     █████╗
██╔═══╝   ╚██╔╝  ██╔══╝  ██║   ██║██║╚██╗██║██║     ██╔══╝  ██╔══██╗██║     ██╔══╝
██║        ██║   ██║     ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝        ╚═╝   ╚═╝      ╚═════╝ ╚═╝  ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝

Provides all our the logic behind our threading mechanism.

Note

Our threads submodules or class does not extends the threading module. They are jus there to clarify our workflow for future contributors :-)

Author:: Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
Special thanks:: https://pyfunceble.github.io/#/special-thanks
Contributors:: https://pyfunceble.github.io/#/contributors
Project link:: https://github.com/funilrys/PyFunceble
Project documentation:: https://pyfunceble.readthedocs.io/en/dev/
Project homepage:: https://pyfunceble.github.io/

License:

Copyright 2017, 2018, 2019, 2020 Nissar Chababy

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.