Console.log slow query warning

[mariusandra]

When developing locally (with postgres), I see stuff like this in the logs way too often:

⌛⌛⌛ Postgres slow query warning after 30 sec {
  queryTextOrConfig: 'INSERT INTO posthog_pluginlogentry (id, team_id, plugin_id, plugin_config_id, timestamp, source,type, message, instance_id) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)',
  values: [
    '01799986-7949-0000-cccc-a3047fc031c1',
    1,
    3,
    10,
    '2021-05-23 13:59:57.513',
    'CONSOLE',
    'LOG',
    'retrying job!',
    '01799986-30dc-0000-cf62-e49fee9f4eff'
  ]
}
[___2] 16:07:49 🤮 Unhandled Promise Error!

The unhandled promise error is from here, and the "retrying job!" message was logged from within a simple plugin:

export const jobs = {
    processEvent (event, meta) {
        console.log('retrying job!')
    }
}
export function processEvent (event, { jobs }) {
    console.log('in processEvent')
    jobs.processEvent(event).runIn(3, 'seconds')
    return event
}

I think it got worse with plugin jobs, but that might be a red herring. Possibly with just plugin logs? The console.log line in the plugins is not await-ed obviously, so perhaps that has something to do with it?

This happens often enough, but is hard to replicate reliably. Anyone have any ideas?

[mariusandra]

For more context, I started exploring this when the bigquery export plugin started giving me lines like this:

It was thus hard to verify if the plugin itself works or not. I hope this happens just in local development, and not in production.

[yakkomajuri]

Documenting here that we can probably have some batching mechanism to periodically insert batches into the DB at once. I think this was mentioned elsewhere too

[neilkakkar]

Possibly related to (new) issues with Bigquery plugin as well: PostHog/bigquery-plugin#9

Edit: it's not related at all.