How to filter some illegal data ?

[kobenaxie]

🚀 Feature

Motivation

For example, the sequence is too long(audio), so need to drop this long sequence data, how to implement it ?

Pitch

Alternatives

Additional context

[github-actions]

Hi! thanks for your contribution!, great first issue!

[tchaton]

Hey @kobenaxie.

You can filter those data during the optimization process.

def fn(...):
       data = ....
       if should_keep(data):
            yield data
       else:
            pass

optimize(
     fn=fn
     ...,
)

Another option is to edit the binary files and index.json. I haven't created any tooling to make this edit-ing easy but it shouldn't be too hard.

[tchaton]

@kobenaxie Let me know if you have some issues filtering bad data.

[kobenaxie]

Hey @tchaton , thank you for your reply, and I have adopted the first method you advised, it works fine.

[tchaton]

Thanks @kobenaxie. Let me add a section in the README for it then.

[tchaton]

Hey @kobenaxie: #432. done ;)