PyFunceble.downloader package¶
Submodules¶
PyFunceble.downloader.base module¶
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides the base of all our downloader.
- Author:
- Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- https://pyfunceble.github.io/#/special-thanks
- Contributors:
- https://pyfunceble.github.io/#/contributors
- Project link:
- https://github.com/funilrys/PyFunceble
- Project documentation:
- https://pyfunceble.readthedocs.io/en/latest/
- Project homepage:
- https://pyfunceble.github.io/
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-
class
PyFunceble.downloader.base.
DownloaderBase
[source]¶ Bases:
object
Provides the base of all downloader classes.
The interface is actually simple, but the part which may be hard to understand is the “downtime” part. What we do, is that we save the download time inside a JSON file, so this class provides the base around the download mechanism but also the generation or update of that JSON file.
-
DOWNLOAD_FREQUENCY
= 1¶ The download frequency (in day).
- Example:
- if 1 is given, it’s once every 24 hours.
Warning
A frequency of
0
or a negative number will force the download every hour.
-
DOWNTIME_INDEX
= None¶ Used to set/track the download time of the current file.
-
all_downtimes
= {'iana': {'iso': '2023-05-29T10:02:39.100379', 'timestamp': 1685354559.100379}, 'user_agents': {'iso': '2023-05-29T10:02:39.221362', 'timestamp': 1685354559.221362}}¶ Stores the download time of all files (self managed).
Provides the authorization to start the download.
-
destination
¶ Provides the current state of the
_destination
attribute.
-
dict_helper
= <PyFunceble.helpers.dict.DictHelper object>¶
-
download_link
¶ Provides the current state of the
_download_link
attribute.
-
is_last_download_expired
() → bool[source]¶ Checks if the last downloaded file is expired (if exists).
-
set_current_downtime
() → PyFunceble.downloader.base.DownloaderBase[source]¶ Sets the current datetime into our registry.
-
set_destination
(value: str) → PyFunceble.downloader.base.DownloaderBase[source]¶ Sets the destination.
Parameters: value – The value to set.
-
PyFunceble.downloader.exceptions module¶
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides the exceptions related to the downloader.
- Author:
- Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- https://pyfunceble.github.io/#/special-thanks
- Contributors:
- https://pyfunceble.github.io/#/contributors
- Project link:
- https://github.com/funilrys/PyFunceble
- Project documentation:
- https://pyfunceble.readthedocs.io/en/latest/
- Project homepage:
- https://pyfunceble.github.io/
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-
exception
PyFunceble.downloader.exceptions.
NoDownloadDestinationGiven
[source]¶ Bases:
PyFunceble.downloader.exceptions.PyFuncebleDownloaderException
Describes the fact that the download destination was not declared.
-
exception
PyFunceble.downloader.exceptions.
NoDownloadLinkGiven
[source]¶ Bases:
PyFunceble.downloader.exceptions.PyFuncebleDownloaderException
Describes the fact that no download link was declared.
-
exception
PyFunceble.downloader.exceptions.
PyFuncebleDownloaderException
[source]¶ Bases:
PyFunceble.exceptions.PyFuncebleException
Describes the downloader (related) exceptions.
PyFunceble.downloader.iana module¶
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides the downloader of the latest iana database file.
- Author:
- Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- https://pyfunceble.github.io/#/special-thanks
- Contributors:
- https://pyfunceble.github.io/#/contributors
- Project link:
- https://github.com/funilrys/PyFunceble
- Project documentation:
- https://pyfunceble.readthedocs.io/en/latest/
- Project homepage:
- https://pyfunceble.github.io/
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-
class
PyFunceble.downloader.iana.
IANADownloader
[source]¶ Bases:
PyFunceble.downloader.base.DownloaderBase
Provides the downloader of our iana file.
-
DOWNLOAD_FREQUENCY
= 1¶
-
DOWNTIME_INDEX
= 'iana'¶
Provides the authorization to start the download.
-
PyFunceble.downloader.ipv4_reputation module¶
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides the downloader of the latest user agents database file.
- Author:
- Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- https://pyfunceble.github.io/#/special-thanks
- Contributors:
- https://pyfunceble.github.io/#/contributors
- Project link:
- https://github.com/funilrys/PyFunceble
- Project documentation:
- https://pyfunceble.readthedocs.io/en/latest/
- Project homepage:
- https://pyfunceble.github.io/
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-
class
PyFunceble.downloader.ipv4_reputation.
IPV4ReputationDownloader
[source]¶ Bases:
PyFunceble.downloader.base.DownloaderBase
Provides the downloader of our user agent file.
-
DOWNLOAD_FREQUENCY
= 1¶
-
DOWNTIME_INDEX
= 'ipv4_reputation'¶
Provides the authorization to start the download.
-
PyFunceble.downloader.public_suffix module¶
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides the downloader of the latest iana database file.
- Author:
- Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- https://pyfunceble.github.io/#/special-thanks
- Contributors:
- https://pyfunceble.github.io/#/contributors
- Project link:
- https://github.com/funilrys/PyFunceble
- Project documentation:
- https://pyfunceble.readthedocs.io/en/latest/
- Project homepage:
- https://pyfunceble.github.io/
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-
class
PyFunceble.downloader.public_suffix.
PublicSuffixDownloader
[source]¶ Bases:
PyFunceble.downloader.base.DownloaderBase
Provides the downloader of our public suffix file.
-
DOWNLOAD_FREQUENCY
= 1¶
-
DOWNTIME_INDEX
= 'psl'¶
Provides the authorization to start the download.
-
PyFunceble.downloader.user_agents module¶
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides the downloader of the latest user agents database file.
- Author:
- Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- https://pyfunceble.github.io/#/special-thanks
- Contributors:
- https://pyfunceble.github.io/#/contributors
- Project link:
- https://github.com/funilrys/PyFunceble
- Project documentation:
- https://pyfunceble.readthedocs.io/en/latest/
- Project homepage:
- https://pyfunceble.github.io/
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-
class
PyFunceble.downloader.user_agents.
UserAgentsDownloader
[source]¶ Bases:
PyFunceble.downloader.base.DownloaderBase
Provides the downloader of our user agent file.
-
DOWNLOAD_FREQUENCY
= 1¶
-
DOWNTIME_INDEX
= 'user_agents'¶
Provides the authorization to start the download.
-
Module contents¶
The tool to check the availability or syntax of domain, IP or URL.
██████╗ ██╗ ██╗███████╗██╗ ██╗███╗ ██╗ ██████╗███████╗██████╗ ██╗ ███████╗
██╔══██╗╚██╗ ██╔╝██╔════╝██║ ██║████╗ ██║██╔════╝██╔════╝██╔══██╗██║ ██╔════╝
██████╔╝ ╚████╔╝ █████╗ ██║ ██║██╔██╗ ██║██║ █████╗ ██████╔╝██║ █████╗
██╔═══╝ ╚██╔╝ ██╔══╝ ██║ ██║██║╚██╗██║██║ ██╔══╝ ██╔══██╗██║ ██╔══╝
██║ ██║ ██║ ╚██████╔╝██║ ╚████║╚██████╗███████╗██████╔╝███████╗███████╗
╚═╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═══╝ ╚═════╝╚══════╝╚═════╝ ╚══════╝╚══════╝
Provides all downloaders.
- Author:
- Nissar Chababy, @funilrys, contactTATAfunilrysTODTODcom
- Special thanks:
- https://pyfunceble.github.io/#/special-thanks
- Contributors:
- https://pyfunceble.github.io/#/contributors
- Project link:
- https://github.com/funilrys/PyFunceble
- Project documentation:
- https://pyfunceble.readthedocs.io/en/latest/
- Project homepage:
- https://pyfunceble.github.io/
License:
Copyright 2017, 2018, 2019, 2020, 2022, 2023 Nissar Chababy
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.