Skip to content

Dtle:When create a schema-table level job with Regular expression, and Src has two schema,job dead  #481

Closed
@yaqigithub

Description

@yaqigithub

Description

When create two schema in Src ,create a Regular expression with Regular expression,the job dead.
When only one schema in Src,job running but can not create new shcema.

Steps to reproduce the issue

  1. create two schema in Src mysql and create table in schema
    create database demo1;
    use demo1;
    create table `sbtest1`( `id` int(7) NOT NULL, `title` VARCHAR(40) NOT NULL, PRIMARY KEY(`id`));
    create table `sbtest2`( `id` int(7) NOT NULL, `title` VARCHAR(40) NOT NULL, PRIMARY KEY(`id`));
    create database demo2;
    use demo2;
    create table `sbtest1`( `id` int(7) NOT NULL, `title` VARCHAR(40) NOT NULL, PRIMARY KEY(`id`));
    create table `sbtest2`( `id` int(7) NOT NULL, `title` VARCHAR(40) NOT NULL, PRIMARY KEY(`id`));

  2. create job
    curl -H "Accept:application/json" -XPOST "http://10.186.62.12:9190/v1/jobs" -d @job.json -s | jq
    issue_job.json.txt

  3. view the job status
    curl -XGET "10.186.62.12:9190/v1/jobs" -s | jq '.[] | .ID, .Status'

4.delete job and delete one schema
drop database demo2;
5. rebuild job
job is running
6.create new schema in Src
create database demo2;
result: Invalid command and job is running

Describe the results you received

job dead

Describe the results you expected

job running and copy normal

Output of ./dtle version:**

Dtle 2.19.07.0 (git: 2.19.07.0 ebdb31e4cec4ca766ca5b7c01b38eb0188873f71)

Additional information

two error Configuration
1、

{
  "TableSchemaRegex": "(\\w*)test(\\w*)",
  "TableSchemaRename": "demo${2}",
  "Tables": [
    {
      "TableName": "sbtest1"
    }
  ]
}
{
  "TableSchemaRegex": "(\\w*)demo(\\w*)",
  "TableSchemaRename": "TEST${2}",
  "Tables": [
    {
      "TableRegex": "(\\w*)sbtest(\\w*)",
      "TableRename": "demo${2}",
      "where": ""
    }
  ]
}

Additional details (log, config, job config etc):

1、

2019/08/08 13:49:26.550319 [DEBUG] setState 3 
2019/08/08 13:49:26.550343 [DEBUG] setState.SaveState 
2019/08/08 13:49:26.550339 [INFO] [jobid:8ef83e27-704e-a1e4-7da2-cde2f97bd9e3] mysql.applier: Apply binlog events to 172.100.9.2.3306 
2019/08/08 13:49:26.550511 [DEBUG] Worker.SaveState: lock: 0xc000ef57a0, 0xc000deb300 
...skipping...
2019/08/08 13:49:36.690770 [ERR] agent: Task "Src" for alloc "a1169d2d-e3ba-9cc7-8b1e-fa5fd26812cd" failed: Wait returned exit code 2, and error Table  conf
iguration error.  
2019/08/08 13:49:36.690783 [INFO] agent: Not restarting task: Src for alloc: a1169d2d-e3ba-9cc7-8b1e-fa5fd26812cd  
2019/08/08 13:49:36.690791 [DEBUG] setState restart 1 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions