We should include a 'sample data option' in input

[concurrent-ai]

Let's say there is a bucket with millions of files - sending that whole list through the partitioner and then pushing it through the DAG during development may be too much.

Perhaps we should include an option to concurrent_core.list that says - 'sample='. Default, it this option is not present is list all files. Sample should cause the partitioner and the subsequent stages of DAG processing to only deal with objects.