SalesforceSource

class

class SalesforceSource(
    username: str | HashiCorpSecret | DirectSecret | EnvironmentSecret,
    password: str | HashiCorpSecret | DirectSecret | EnvironmentSecret,
    security_token: str | HashiCorpSecret | DirectSecret | EnvironmentSecret,
    query: str | List[str],
    instance_url: str = None,
    include_deleted: bool = False,
    initial_last_modified: str = None,
    api_version: str = None,
)

Bases: SourcePlugin

Categories: source

Parent class for input plugins.

Methods: chunk(working_dir: str) -> Union[str, Tuple[str, ...], List[str]] Trigger the import of the data. The method will receive a folder where it must store the data as parquet files, and return a list of the paths of the files created. This files will then be loaded and mapped to the dataset function in positional order, so if you want file.parquet to be the first argument of the dataset function, you must return it first. If you want a parameter to receive multiple files, return a list of the paths. For example, you would give the following return to provide a first argument with a single file and a second argument with two files: return ["file1.parquet", ["file2.parquet", "file3.parquet"]]

Initializes the SalesforceSource with the given query(s) and the credentials required to access Salesforce.

Parameters

parameter

usernamestr | HashiCorpSecret | DirectSecret | EnvironmentSecret

The username to access Salesforce.

parameter

passwordstr | HashiCorpSecret | DirectSecret | EnvironmentSecret

The password to access Salesforce.

parameter

security_tokenstr | HashiCorpSecret | DirectSecret | EnvironmentSecret

The security token to access Salesforce.

parameter

querystr | List[str]

The query or queries to execute in Salesforce. It can be a single string or a list of strings.

parameter

instance_urlstr, optional

The URL of the instance to which we want to connect. Only necessary when the username and password are associated to more than one instance. Defaults to None.

parameter

include_deletedbool, optional

Whether to include deleted records in the query results. Defaults to False.

parameter

initial_last_modifiedstr, optional

The initial last modified date to use in the queries. This is useful when we want to query only the records that have been modified since a certain date. Defaults to None. If provided, it must be a string in Salesforce datetime format (with informed timezone) and the query must contain the token $lastModified, which will be replaced by the latest 'last_modified' value in each execution.

parameter

api_versionstr, optional

The Salesforce API version to use. Defaults to None, which will default to the latest version supported by simple_salesforce.

Properties

property

initial_valuesdict

Return a dictionary with the initial values to be stored after execution of the plugin. They will be accessible in the next execution of the plugin. The dictionary must have the parameter names as keys and the initial values as values, all the type string.

Returns: dict: A dictionary with the initial values of the parameters of the plugin.

property

passwordHashiCorpSecret | DirectSecret | EnvironmentSecret

property

queryList[str]

property

security_tokenHashiCorpSecret | DirectSecret | EnvironmentSecret

property

usernameHashiCorpSecret | DirectSecret | EnvironmentSecret

Methods

method

chunk

def chunk(working_dir: str) -> List[str]

Trigger the import of the data. This must be implemented in any class that inherits from this class unless directly implementing streaming. The method will receive a folder where it must store the data as parquet files, and return a list of the paths of the files created. This files will then be loaded and mapped to the dataset function in positional order, so if you want file.parquet to be the first argument of the dataset function, you must return it first. If you want a parameter to receive multiple files, return a list of the paths. For example, you would give the following return to provide a first argument with a single file and a second argument with two files: return ["file1.parquet", ["file2.parquet", "file3.parquet"]]

Parameters:

parameter

working_dirstr

The folder where the files must be stored

Returns:

Union[str, Tuple[str, ...], List[str]]: The path of the file(s) created, in the order they must be mapped to the dataset function