Welcome to Smoke Forcing’s documentation!

smoke_forcing

smoke_forcing package

Subpackages

smoke_forcing.utils package
Submodules
smoke_forcing.utils.conf_tools module
exception smoke_forcing.utils.conf_tools.InvalidConfigException[source]

Bases: Exception

smoke_forcing.utils.conf_tools.check_config(func)[source]

Wrapper function that checks whether the config parameter passed to the function wrapped is:

  1. A dict

2) str or Path. In this case, it will check if the path is a directory, and if it exists. If the config can be opened, it is loaded into a dict, and fed to the function wrapped in place of the str or Path

1) The function wrapped must have config as a positional argument. The position of the parameter itself is of no consequence.

Parameters:func – python function to be wrapped
Returns:whatever the wrapped function returns
smoke_forcing.utils.download module
smoke_forcing.utils.ftp module
smoke_forcing.utils.url_downloader module
smoke_forcing.utils.url_parse module
smoke_forcing.utils.url_parse.filter_by_ext(url_list, ext)[source]

Filter list by extension

Args:
url_list (list of str) ext (str): File extension to filter by
Returns:
[type]: [description]
smoke_forcing.utils.url_parse.get_auth(username, password)[source]
smoke_forcing.utils.url_parse.get_url_list(base_url, soup, container='a')[source]

Get all urls from webpage in container type container

Args:
base_url (str) soup (BeautifulSoup): parsed webpage container (str, optional): What HTML container to look for hrefs in. Defaults to “a”.
Returns:
[type]: [description]
smoke_forcing.utils.url_parse.parse_protected_html(url, username, password)[source]

Parse password protected site

Args:
url (str): url for secure site username (str) password (str)
Returns:
BeautifulSoup: parsed HTML
Module contents

Submodules

smoke_forcing.daemon module

smoke_forcing.downloaders module

smoke_forcing.main module

smoke_forcing.urls module

Module contents

Copyright 2020 Ashutosh Bhudia, Brandon Dos Remedios

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Indices and tables