Setting the cursor value for stream in UI/API

[pcothenet]

Tell us about the problem you're trying to solve

What are you trying to do, and why is it hard? A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

I have a table that's too big and somehow fails the postgres to BigQuery replication (a problem I would like to fix, but that's different).

The best way I've found to fix this is avoid replicating older records by setting an initial value for the cursor. I do this by directly updating the state table in the airbyte postgres backend.

However, every time I re-sync the full schema, I have to reset that timestamp manually

Describe the solution you’d like

• Either be able to set the cursor via UI or API
• Or be able to set a minCursor (or initialCursor) on the stream, to avoid bothering with other records.

Describe the alternative you’ve considered or used

Currently, I'm working around this by overwriting the state record for this connection in the state table (by connecting to the pg backend). Not desirable as it is very error-prone.

Additional context

I'll file another ticket about my pg replication issues, because that would honestly solve my need for this.

Are you willing to submit a PR?

Probably more time than I can afford here ;-)

[marcosmarxm]

Thanks for the suggestion @pcothenet this really would be valuable for database syncs.

[jrhizor]

duplicate of #3092