-
Notifications
You must be signed in to change notification settings - Fork 447
chore(llmobs/langchain): stop using in-process vcr #13935
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
1945240
to
07862ad
Compare
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 286 ± 6 ms. The average import time from base is: 286 ± 6 ms. The import time difference between this PR and base is: -0.8 ± 0.3 ms. Import time breakdownThe following import paths have shrunk:
|
BenchmarksBenchmark execution time: 2025-07-10 14:27:38 Comparing candidate commit a5a7065 in PR branch Found 0 performance improvements and 2 performance regressions! Performance is the same for 516 metrics, 2 unstable metrics. scenario:iastaspects-lstrip_aspect
scenario:iastaspectsospath-ospathsplitext_aspect
|
I suspect that the in-process VCR is leading to flaky tests as: - VCR patches all http connections including possibly those to the test agent; and - I don't trust in-process VCR monkeypatching very much Instead, we can continue to use VCR but out of process by proxying through the test agent which now supports a VCR feature.
07862ad
to
a5a7065
Compare
I suspect that the in-process VCR is leading to flaky tests as:
Instead, we can continue to use VCR but out of process by proxying through the test agent which now supports a VCR feature.
Checklist
Reviewer Checklist