[Bug]: Fabric Dependency Issues with gRPC and Google Cloud Libraries #747

joshuafuller · 2024-07-18T19:39:47Z

What happened?

What happened?

I was using the fabric package for various commands, such as changing the default model and summarizing YouTube videos. Despite multiple attempts to reinstall fabric and its dependencies, I am still encountering warning messages related to grpcio and other related libraries. The specific warnings are:

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721329796.596775 48349 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.

These warnings appear consistently during different fabric operations, not just when changing the default model.

Environment:

Operating System: Windows 11 with WSL2
Python Version: 3.10
Fabric Version: 1.4.0
Latest Commit: d75ea47
Git Version: 2.34.1
Dependency Versions:
- grpcio==1.65.1
- google-cloud-aiplatform==1.59.0
- google-cloud-storage==2.17.0

Steps Taken to Resolve the Issue:

Full Uninstall and Reinstallation:
- Uninstalled fabric using pipx: pipx uninstall fabric
- Deleted the local fabric repository: rm -rf ~/github/fabric
- Re-cloned the fabric repository from GitHub: git clone https://github.com/danielmiessler/fabric.git
- Reinstalled fabric using pipx: pipx install fabric
- Re-setup fabric as a new installation: fabric --setup
- I performed these steps twice to ensure a clean installation.
Dependency Verification:
- Checked installed versions of key dependencies using: pipx runpip fabric show grpcio
- Noted the installed versions:
  - grpcio==1.65.1
  - google-cloud-aiplatform==1.59.0
  - google-cloud-storage==2.17.0
- Upgraded dependencies: pipx runpip fabric install --upgrade grpcio google-cloud-aiplatform google-cloud-storage

Example Commands Producing Warnings:

Changing the default model:

fabric --changeDefaultModel gpt-4o-mini
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721329796.587000   48349 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0000 00:00:1721329796.596775   48349 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.
Default model changed to gpt-4o-mini. Please restart your terminal to use it.

Summarizing a YouTube video (snipped for brevity):

yt https://www.youtube.com/watch?v=_CwGUt0CpvU | summarize -s
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721330542.882595   52151 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0000 00:00:1721330542.888972   52151 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.
[SNIPPED FOR BREVITY - Normal Output]

The command still provided the expected output after the warning messages.

Version check

Yes I was.

Relevant log output

fabric --changeDefaultModel gpt-4o-mini
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721329796.587000   48349 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0000 00:00:1721329796.596775   48349 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.
Default model changed to gpt-4o-mini. Please restart your terminal to use it.


yt https://www.youtube.com/watch?v=_CwGUt0CpvU | summarize -s
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721330542.882595   52151 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0000 00:00:1721330542.888972   52151 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.

Relevant screenshots (optional)

The text was updated successfully, but these errors were encountered:

manojkollam · 2024-07-19T00:25:47Z

I am getting the same type of behavior on a fresh install on ubuntu on WSL

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721348702.029898 172323 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
I0000 00:00:1721348702.035481 172323 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.

joshuafuller · 2024-07-19T19:49:12Z

I found a temporary fix for the issue:

To mitigate the problem, run the following commands:

pipx runpip fabric uninstall grpcio
pipx runpip fabric install grpcio==1.64.0

This resolves the warning messages related to grpcio and other related libraries in my environment.

For more details on the grpc bug, see:

As for @07pepa's comments, they were completely useless and off-topic. They didn't help at all and only added noise to the conversation. If you can't contribute constructively, please refrain from commenting.

I hope this helps others facing the same issue.

manojkollam · 2024-07-19T23:53:15Z

I found a temporary fix for the issue:

To mitigate the problem, run the following commands:
pipx runpip fabric uninstall grpcio
pipx runpip fabric install grpcio==1.64.0
This resolves the warning messages related to grpcio and other related libraries in my environment.

For more details on the grpc bug, see:

grpc/grpc#37222

grpc/grpc#37178

As for @07pepa's comments, they were completely useless and off-topic. They didn't help at all and only added noise to the conversation. If you can't contribute constructively, please refrain from commenting.

I hope this helps others facing the same issue.

I can confirm that this fix downgrading grpcio to 1.64 is working for me as well

andeeisaacs · 2024-07-20T01:38:03Z

Also confirming that the uninstall and reinstall of grpcio worked and resolved the warning messages.

Thanks, @joshuafuller.

fearby · 2024-07-21T08:54:12Z

MacOS same problem and fix doesn't work

ph3ll3r · 2024-07-24T21:34:22Z

[[Bug]: Fabric Dependency Issues with gRPC and Google Cloud Libraries] #747 (comment))
#747

I only want to use Google's models/gemini-1.5-pro and ollama models. I want exclude for the time being OpenAI (fabric default) and Claude.ai.

After a reboot, I get this error "Error: Client error '401 Unauthorized' for url 'https://api.openai.com/v1/models'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401"

I followed the suggested format a similar way described by Joshua Fuller

Full Uninstall and Reinstallation:

Uninstalled fabric using pipx: pipx uninstall fabric
Deleted the local fabric repository: rm -rf ~/github/fabric
Re-cloned the fabric repository from GitHub: git clone https://github.com/danielmiessler/fabric.git
Reinstalled fabric using pipx: pipx install fabric
Re-setup fabric as a new installation: fabric --setup

Bypassed the openai key and claude.ai key
Inserted Google generative ai api key and youtube api key

I performed these steps twice to ensure a clean installation.

I tested that fabric was installed properly:

fabric --listmodels and found models/gemini-1.5-pro
fabric --changeDefaultModel models/gemini-1.5-pro
At this point the gRPC error, was still occurring but I was getting answers from fabric

Once, I reboot OpenAI took precedence and returned the error "Error: Client error '401 Unauthorized' for url 'https://api.openai.com/v1/models'". This reminds me of the "Harry Belafonte & Odetta - A Hole in the Bucket: "https://www.youtube.com/watch?v=xVAvMIhvqfk"

There needs to be a way to persist fabric --changeDefaultModel.

Also, add as many keys as you like but have a way to reset a combination of keys which are dynamically persisted but can be changed and restart the shell and happy fabric-ing to handle as an example (Fabric: This OPENSOURCE AI Framework can AUTOMATE YOUR LIFE & ANY TASK (Setup with Ollama & Groq) : "https://www.youtube.com/watch?v=Iup3-Jvkrhg" ) as a permutation of the ability to reset a combination of keys

joshuafuller · 2024-07-25T13:51:30Z

@ph3ll3r, it looks like your issue differs from the one I originally reported. You might want to start a new thread to address your specific problem.

The error I encountered affected all Fabric operations and involved warnings related to grpcio and other libraries, such as:

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1721329796.596775   48349 check_gcp_environment.cc:61] BIOS data file does not exist or cannot be opened.

These warnings occurred during various commands, including changing the default model and summarizing YouTube videos.

Your error seems to be a 401 Unauthorized error related to OpenAI's API after a reboot, which suggests a different underlying issue.

Please refer to the photo I uploaded to see the differences in our issues.

th3f001 · 2024-07-27T19:57:49Z

I found a temporary fix for the issue:

To mitigate the problem, run the following commands:
pipx runpip fabric uninstall grpcio
pipx runpip fabric install grpcio==1.64.0
This resolves the warning messages related to grpcio and other related libraries in my environment.

For more details on the grpc bug, see:

grpc/grpc#37222

grpc/grpc#37178

As for @07pepa's comments, they were completely useless and off-topic. They didn't help at all and only added noise to the conversation. If you can't contribute constructively, please refrain from commenting.

I hope this helps others facing the same issue.

On MacOS Sonoma 14.5.1 the reported fix works. Thanks!

ph3ll3r · 2024-07-28T16:09:03Z

I used this and it did not work.

Dependency Verification:

Checked installed versions of key dependencies using: pipx runpip fabric show grpcio

Noted the installed versions:

grpcio==1.65.1 fails but 1.64 works
google-cloud-aiplatform==1.59.0
google-cloud-storage==2.17.0
Upgraded dependencies: pipx runpip fabric install --upgrade grpcio google-cloud-aiplatform google-cloud-storage

th3f001 commented 20 hours ago

To mitigate the problem, run the following commands:

worked perfectly

pipx runpip fabric uninstall grpcio
pipx runpip fabric install grpcio==1.64.0

fcalazans · 2024-08-03T20:57:08Z

Having the same problem with the initial message of the thread:

❯ fabric --listmodels
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1722717785.434341  880089 check_gcp_environment_no_op.cc:29] ALTS: Platforms other than Linux and Windows are not supported
I0000 00:00:1722717786.424544  880089 check_gcp_environment_no_op.cc:29] ALTS: Platforms other than Linux and Windows are not supported

Device:
MacBook Pro
14-inch, 2023
Chip: Apple M2 Pro
Memory: 16 GB
macOS: 14.6 (23G80)

worked perfectly

pipx runpip fabric uninstall grpcio
pipx runpip fabric install grpcio==1.64.

joshuafuller added the bug Something isn't working label Jul 18, 2024

This comment has been minimized.

Sign in to view

QCadjunct mentioned this issue Jul 25, 2024

[Bug]: Being able to persist my default model which is not the OpenAI key (fabric default) #767

Closed

1 task

justinlinw mentioned this issue Aug 16, 2024

[feat] Add support for running CyBench without HELM API andyzorigin/cybench#583

Merged

danielmiessler closed this as completed Aug 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Fabric Dependency Issues with gRPC and Google Cloud Libraries #747

[Bug]: Fabric Dependency Issues with gRPC and Google Cloud Libraries #747

joshuafuller commented Jul 18, 2024 •

edited

Loading

manojkollam commented Jul 19, 2024

joshuafuller commented Jul 19, 2024

manojkollam commented Jul 19, 2024

andeeisaacs commented Jul 20, 2024

This comment has been minimized.

fearby commented Jul 21, 2024

ph3ll3r commented Jul 24, 2024 •

edited

Loading

joshuafuller commented Jul 25, 2024

th3f001 commented Jul 27, 2024

ph3ll3r commented Jul 28, 2024

fcalazans commented Aug 3, 2024

[Bug]: Fabric Dependency Issues with gRPC and Google Cloud Libraries #747

[Bug]: Fabric Dependency Issues with gRPC and Google Cloud Libraries #747

Comments

joshuafuller commented Jul 18, 2024 • edited Loading

What happened?

Version check

Relevant log output

Relevant screenshots (optional)

manojkollam commented Jul 19, 2024

I am getting the same type of behavior on a fresh install on ubuntu on WSL

joshuafuller commented Jul 19, 2024

manojkollam commented Jul 19, 2024

andeeisaacs commented Jul 20, 2024

This comment has been minimized.

fearby commented Jul 21, 2024

ph3ll3r commented Jul 24, 2024 • edited Loading

joshuafuller commented Jul 25, 2024

th3f001 commented Jul 27, 2024

ph3ll3r commented Jul 28, 2024

I used this and it did not work.

Dependency Verification:

Checked installed versions of key dependencies using: pipx runpip fabric show grpcio

Noted the installed versions:

th3f001 commented 20 hours ago

To mitigate the problem, run the following commands:

worked perfectly

fcalazans commented Aug 3, 2024

worked perfectly

joshuafuller commented Jul 18, 2024 •

edited

Loading

ph3ll3r commented Jul 24, 2024 •

edited

Loading