Skip to content

Main CLI application

This section contains the API reference for the main CLI application.

Overview

This module contains the main CLI (Typer) app.

The app is defined in the app variable. The main function is the entry point for the CLI app.

APP_NAME = 'opendata-pipeline' module-attribute

Application name for the CLI app.

app = typer.Typer(name=APP_NAME, help="A command line tool for running the UK-IPOP Medical Examiner's Open Data Pipeline.", rich_markup_mode='rich') module-attribute

The Typer CLI app.

analyze(use_remote=typer.Option(False, help='Whether to use the remote configuration or not. Default is False (i.e. use local config.json)'), update_remote=typer.Option(False, help='Whether to update the remote configuration or not. Default is False (i.e. update local config.json)'))

:warning: Analyze the data.

This command takes the various output files and joins them to the original data for a "wide-form" datafile that is commonly used in data analysis.

This command will geocode data sources and save it to the data directory.

If use_remote is True, the remote configuration will be used. Otherwise, the local configuration will be used. Configuration file required for

Unless you are me, you cannot update_remote because you do not have the correct permissions.

Expects the data to be fetched before running this command. Expects the drugs to be extracted before running this command. Expects the data to be geocoded before running this command.

Example: opendata-pipeline analyze --use-remote

Source code in opendata_pipeline/main.py 
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
@app.command("analyze")
def analyze(
    use_remote: bool = typer.Option(
        False,
        help="Whether to use the remote configuration or not. Default is False (i.e. use local config.json)",
    ),
    update_remote: bool = typer.Option(
        False,
        help="Whether to update the remote configuration or not. Default is False (i.e. update local config.json)",
    ),
) -> None:
    """:warning: Analyze the data.

    This command takes the various output files and joins them to the original data for
    a "wide-form" datafile that is commonly used in data analysis.

    This command will geocode data sources and save it to the data directory.

    If `use_remote` is True, the remote configuration will be used. Otherwise, the local configuration will be used.
    Configuration file required for

    Unless you are me, you cannot `update_remote` because you do not have the correct permissions.

    Expects the data to be fetched before running this command.
    Expects the drugs to be extracted before running this command.
    Expects the data to be geocoded before running this command.

    Example: opendata-pipeline analyze --use-remote
    """
    utils.console.rule("[bold cyan]Analyzing data")
    settings = get_settings(remote=use_remote)
    analyzer.run(settings=settings)
    utils.console.log("[bold green]Analysis complete!")

extract_drugs(use_remote=typer.Option(False, help='Whether to use the remote configuration or not. Default is False (i.e. use local config.json)'))

Extract drugs from data sources.

This command will extract drugs from data sources and save it to the data directory.

** You will need the extract-drugs CLI program installed and in your PATH for this command to work. You can get it here: https://github.com/UK-IPOP/drug-extraction

If use_remote is True, the remote configuration will be used. Otherwise, the local configuration will be used.

Expects the data to be fetched before running this command.

Example: opendata-pipeline extract-drugs --use-remote

Source code in opendata_pipeline/main.py 
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
@app.command("extract-drugs")
def extract_drugs(
    use_remote: bool = typer.Option(
        False,
        help="Whether to use the remote configuration or not. Default is False (i.e. use local config.json)",
    )
) -> None:
    """Extract drugs from data sources.

    This command will extract drugs from data sources and save it to the data directory.

    ** You will need the `extract-drugs` CLI program installed and in your PATH for this command to work.
    You can get it here: https://github.com/UK-IPOP/drug-extraction

    If `use_remote` is True, the remote configuration will be used. Otherwise, the local configuration will be used.

    Expects the data to be fetched before running this command.

    Example: opendata-pipeline extract-drugs --use-remote
    """
    utils.console.rule("[bold cyan]Extracting drugs")
    settings = get_settings(remote=use_remote)
    drug_extractor.run(settings=settings)
    utils.console.log("[bold green]Drug extraction complete!")

fetch(use_remote=typer.Option(False, help='Whether to use the remote configuration or not. Default is False (i.e. use local config.json)'), update_remote=typer.Option(False, help='Whether to update the remote configuration or not. Default is False (i.e. update local config.json)'))

:warning: Fetch data from data sources.

This command will fetch data from data sources and save it to the data directory.

If use_remote/update_remote are True, the remote configuration will be used/updated. Otherwise, the local configuration will be used.

If you are not me, you cannot update_remote because you do not have the correct permissions.

Expects the project to be initialized before running this command.

Example: opendata-pipeline fetch --use-remote

Source code in opendata_pipeline/main.py 
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
@app.command("fetch")
def fetch(
    use_remote: bool = typer.Option(
        False,
        help="Whether to use the remote configuration or not. Default is False (i.e. use local config.json)",
    ),
    update_remote: bool = typer.Option(
        False,
        help="Whether to update the remote configuration or not. Default is False (i.e. update local config.json)",
    ),
) -> None:
    """:warning: Fetch data from data sources.

    This command will fetch data from data sources and save it to the data directory.

    If `use_remote`/`update_remote` are True, the remote configuration will be used/updated. Otherwise, the local configuration will be used.

    If you are not me, you cannot `update_remote` because you do not have the correct permissions.

    Expects the project to be initialized before running this command.

    Example: opendata-pipeline fetch --use-remote
    """
    utils.console.rule("[bold cyan]Fetching data")
    settings = get_settings(remote=use_remote)
    asyncio.run(fetcher.run(settings=settings, update_remote=update_remote))
    utils.console.log("[bold green]Data fetching complete")

geocode(use_remote=typer.Option(False, help='Whether to use the remote configuration or not. Default is False (i.e. use local config.json)'), custom_key=typer.Option(None, help='Your own ArcGIS API key, geocoding not possible otherwise.'))

:warning: Geocode data sources.

This command will geocode data sources and save it to the data directory.

If use_remote is True, the remote configuration will be used. Otherwise, the local configuration will be used.

Expects the data to be fetched before running this command.

If you are not me, you must provide your own ArcGIS API key using the custom_key option.

Example: opendata-pipeline geocode --use-remote

Source code in opendata_pipeline/main.py 
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
@app.command("geocode")
def geocode(
    use_remote: bool = typer.Option(
        False,
        help="Whether to use the remote configuration or not. Default is False (i.e. use local config.json)",
    ),
    custom_key: Optional[str] = typer.Option(
        None, help="Your own ArcGIS API key, geocoding not possible otherwise."
    ),
) -> None:
    """:warning: Geocode data sources.

    This command will geocode data sources and save it to the data directory.

    If `use_remote` is True, the remote configuration will be used. Otherwise, the local configuration will be used.

    Expects the data to be fetched before running this command.

    If you are not me, you must provide your own ArcGIS API key using the `custom_key` option.

    Example: opendata-pipeline geocode --use-remote
    """
    utils.console.rule("[bold cyan]Geocoding data")
    settings = get_settings(remote=use_remote)
    asyncio.run(geocoder.run(settings=settings, alternate_key=custom_key))
    utils.console.log("[bold green]Geocoding complete!")

get_settings(remote)

Get the settings for the app.

Parameters:

Name Type Description Default
remote bool

Whether or not to use the remote config.

 required

Returns:

Type Description
models.Settings

models.Settings: The settings for the app.
Source code in opendata_pipeline/main.py 
38
39
40
41
42
43
44
45
46
47
48
49
def get_settings(remote: bool) -> models.Settings:
    """Get the settings for the app.

    Args:
        remote (bool): Whether or not to use the remote config.

    Returns:
        models.Settings: The settings for the app.
    """
    if remote:
        return manage_config.get_remote_config()
    return manage_config.get_local_config()

init()

Initialize the project by creating the data directory.

Source code in opendata_pipeline/main.py 
52
53
54
55
56
57
@app.command("init")
def init():
    """Initialize the project by creating the data directory."""
    utils.console.rule("[bold cyan]Initializing project")
    utils.setup()
    utils.console.log("[bold green]Project initialization complete")

teardown()

Teardown the project, deleting all data files.

Source code in opendata_pipeline/main.py 
178
179
180
181
182
183
@app.command("teardown")
def teardown():
    """Teardown the project, deleting all data files."""
    utils.console.rule("[bold cyan]Tearing down project")
    utils.teardown()
    utils.console.log("[bold green]Project torn down successfully")