ReIndex blob data files directly from Azure

[professorDante]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Place new files into the Blob storage from the resource group directly in Azure, and be able to reindex vector DB.

Any log messages given by the failure

Expected/desired behavior

see above

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)
N/A

azd version?

run azd version and copy paste here.
N/A

Versions

Mention any other details that might be useful

It would be most useful to be able to reindex the vector DB if you have pushed new files into the Blob. You can do this from the prepdocs.py script, if you've done this locally.

However, say we have Azure files and wish to add them to the index, once we drag and drop into the Blob, how do we reindex that proceedure?

---

Thanks! We'll be in touch soon.

[kibnelbachyr]

I am also interested if there is any option to do this from the Azure Portal.

[pamelafox]

I think the approach for this would be to use an Azure Function with a blob trigger, and then use prepdocs.py to process that blob trigger.

For example, here's what the function code might start with:

@app.blob_trigger(arg_name="doc", path="unchunked", connection="AzureWebJobsStorage")
def create_chunks(doc: func.InputStream):
    return asyncio.run(create_chunks_async(doc))

You would need to call prepdocs from that code. It'd take some custom coding and provisioning, but should be possible.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.