Google Cloud services allow you to upload and manipulate Tables as spreadsheets (via GoogleSheets) or query them as SQL database tables (via GoogleBigQuery). You can also upload/store/download them as binary objects (via GoogleCloudStorage). Google also offers an API for civic information using GoogleCivic and admin information using the Google Workspace Admin SDK.
For all of these services you will need to enable the APIs for your Google Cloud account and obtain authentication tokens or other credentials to access them from your scripts. If you are the administrator of your Google Cloud account, you can do both of these at Google Cloud Console APIs and Services. The connectors below have more specific information about how to authenticate.
The GoogleAdmin class allows you to get information about groups and members in Google Admin.
In order to instantiate the class, you must pass Google service account credentials as a dictionary, or store the credentials as a JSON string in the GOOGLE_APPLICATION_CREDENTIALS environment variable. You must also provide an email address for domain-wide delegation.
To instantiate the GoogleAdmin class, you can either pass the constructor a dict containing your Google service account credentials or define the environment variable GOOGLE_APPLICATION_CREDENTIALS to contain a JSON encoding of the dict.
from parsons import GoogleAdmin
# First approach: Use API credentials via environmental variables
admin = GoogleAdmin(None, 'fakeemail@fakedomain.com')
# Second approach: Pass API credentials as argument
credential_filename = 'google_application_credentials.json'
credentials = json.load(open(credential_filename))
sheets = GoogleSheets(credentials, 'fakeemail@fakedomain.com')You can then get information about groups and members using instance methods:
members = admin.get_all_group_members('group_key')
groups = admin.get_all_groups(domain='fakedomain.com').. autoclass:: parsons.google.google_admin.GoogleAdmin :inherited-members:
Google BigQuery is a cloud data warehouse solution. Data is stored in tables, and users can query using SQL. BigQuery uses datasets as top level containers for tables, and datasets are themselves contained within Google Cloud projects.
To instantiate the GoogleBigQuery class, you can pass the constructor a string containing either the name of the Google service account credentials file or a JSON string encoding those credentials. Alternatively, you can set the environment variable GOOGLE_APPLICATION_CREDENTIALS to be either of those strings and call the constructor without that argument.
from parsons import GoogleBigQuery
# Set as environment variable so we don't have to pass it in. May either
# be the file name or a JSON encoding of the credentials.
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'google_credentials_file.json'
bigquery = GoogleBigQuery()Alternatively, you can pass the credentials in as an argument. In the example below, we also specify the project.
# Project in which we're working
project = 'parsons-test'
bigquery = GoogleBigQuery(
app_creds='google_credentials_file.json',
project=project
)We can now upload/query data.
dataset = 'parsons_dataset'
table = 'parsons_table'
# Table name should be project.dataset.table, or dataset.table, if
# working with the default project
table_name = f"`{project}.{dataset}.{table}`"
# Must be pre-existing bucket. Create via GoogleCloudStorage() or
# at https://console.cloud.google.com/storage/create-bucket. May be
# omitted if the name of the bucket is specified in environment
# variable GCS_TEMP_BUCKET.
gcs_temp_bucket = 'parsons_bucket'
# Create dataset if it doesn't already exist
bigquery.client.create_dataset(dataset=dataset, exists_ok=True)
parsons_table = Table([{'name':'Bob', 'party':'D'},
{'name':'Jane', 'party':'D'},
{'name':'Sue', 'party':'R'},
{'name':'Bill', 'party':'I'}])
# Copy table in to create new BigQuery table
bigquery.copy(
table_obj=parsons_table,
table_name=table_name,
tmp_gcs_bucket=gcs_temp_bucket
)
# Select from project.dataset.table
bigquery.query(f'select name from {table_name} where party = "D"')
# Query with parameters
bigquery.query(
f"select name from {table_name} where party = %s",
parameters=["D"]
)
# Delete the table when we're done
bigquery.client.delete_table(table=table_name).. autoclass:: parsons.google.google_bigquery.GoogleBigQuery :inherited-members:
Google Cloud Storage is a cloud file storage system. It uses buckets in which to store arbitrary files referred to as blobs. You may use this connector to upload Parsons tables as blobs, download them to files, and list available blobs.
To use the GoogleCloudStorage class, you will need Google service account credentials. If you are the administrator of your Google Cloud account, you can generate them in the Google Cloud Console APIs and Services.
To instantiate the GoogleBigQuery class, you can pass the constructor a string containing either the name of your Google service account credentials file or a JSON string encoding those credentials. Alternatively, you can set the environment variable GOOGLE_APPLICATION_CREDENTIALS to be either of those strings and call the constructor without that argument.
from parsons import GoogleCloudStorage
# Set as environment variable so we don't have to pass it in. May either
# be the file name or a JSON encoding of the credentials.
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'google_credentials_file.json'
gcs = GoogleCloudStorage()Alternatively, you can pass the credentials in as an argument. In the example below, we also specify the project.
credentials_filename = 'google_credentials_file.json'
project = 'parsons-test' # Project in which we're working
gcs = GoogleCloudStorage(app_creds=credentials_filename, project=project)Now we can create buckets, upload blobs to them and and list/retrieve the available blobs.
gcs.create_bucket('parsons_bucket')
gcs.list_buckets()
gcs.upload_table(bucket='parsons_bucket', table=parsons_table, blob_name='parsons_blob')
gcs.get_blob(bucket_name='parsons_bucket', blob_name='parsons_blob').. autoclass:: parsons.google.google_cloud_storage.GoogleCloudStorage :inherited-members:
Google Civic is an API which provides helpful information about elections. In order to access Google Civic you must create a Google Developer Key in their API console. In order to use Google Civic, you must enable this specific end point.
The Google Civic API utilizes the Voting Information Project to collect key civic information such as personalized ballots and polling location information.
To instantiate the GoogleCivic class, you can pass the constructor a string containing the Google Civic API key you've generated for your project, or set the environment variable GOOGLE_CIVIC_API_KEY to that value.
from parsons import GoogleCivic
# Set as environment variable so we don't have to pass it in. May either
# be the file name or a JSON encoding of the credentials.
os.environ['GOOGLE_CIVIC_API_KEY'] = 'AIzaSyAOVZVeL-snv3vNDUdw6QSiCvZRXk1xM'
google_civic = GoogleCivic()Alternatively, you can pass the credentials in as an argument. In the example below, we also specify the project.
google_civic = GoogleCivic(api_key='AIzaSyAOVZVeL-snv3vNDUdw6QSiCvZRXk1xM')Now you can retrieve election information
elections = google_civic.get_elections()
address = '1600 Pennsylvania Avenue, Washington DC'
election_id = '7000' # General Election
google_civic.get_polling_location(election_id=election_id, address=address)You can also retrieve represntative information such as offices, officals, etc.
address = '1600 Pennsylvania Avenue, Washington DC'
representatives = google_civic.get_representatives_by_address(address=address).. autoclass :: parsons.google.google_civic.GoogleCivic :inherited-members:
The GoogleSheets class allows you to interact with Google service account spreadsheets, called "Google Sheets." You can create, modify, read, format, share and delete sheets with this connector.
In order to instantiate the class, you must pass Google service account credentials as a dictionary, or store the credentials as a JSON file locally and pass the path to the file as a string in the GOOGLE_DRIVE_CREDENTIALS environment variable. You can follow these steps:
- Go to the Google Developer Console and make sure the "Google Drive API" and the "Google Sheets API" are both enabled.
- Go to the credentials page via the lefthand sidebar. On the credentials page, click "create credentials".
- Choose the "Service Account" option and fill out the form provided. This should generate your credentials.
- Select your newly created Service Account on the credentials main page.
- select "keys", then "add key", then "create new key". Pick the key type JSON. The credentials should start to automatically download.
You can now copy and paste the data from the key into your script or (recommended) save it locally as a JSON file.
To instantiate the GoogleSheets class, you can either pass the constructor a dict containing your Google service account credentials or define the environment variable GOOGLE_DRIVE_CREDENTIALS to contain a path to the JSON file containing the dict.
from parsons import GoogleSheets
# First approach: Use API credentials via environmental variables
sheets = GoogleSheets()
# Second approach: Pass API credentials as argument
credential_filename = 'google_drive_service_credentials.json'
credentials = json.load(open(credential_filename))
sheets = GoogleSheets(google_keyfile_dict=credentials)You can then create/modify/retrieve documents using instance methods:
sheet_id = sheets.create_spreadsheet('Voter Cell Phones')
sheets.append_to_sheet(sheet_id, people_with_cell_phones)
parsons_table = sheets.get_worksheet(sheet_id)You may also want to share the document with your service or user account.
.. autoclass:: parsons.google.google_sheets.GoogleSheets :inherited-members: