-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Performance]: Building a 3-layer MLP network with PyTorch, inference slows down after conversion with OpenViNO #22751
Comments
@qiwang067 please share the
https://github.com/qiwang067/openvino_rl/blob/main/dqn.py#L150 returns one additional argument in my env |
@andrei-kochin please check the following link: |
With what reference do you compare? Is it torch eager? I see improved performance in my small script compared to torch eager. Could you provide |
I compared with original torch version, not torch eager. |
@qiwang067 I still see better performance on openvino: import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import openvino as ov
import time
class MLP(nn.Module):
def __init__(self, n_states,n_actions,hidden_dim=128):
super(MLP, self).__init__()
self.fc1 = nn.Linear(n_states, hidden_dim)
self.fc2 = nn.Linear(hidden_dim,hidden_dim)
self.fc3 = nn.Linear(hidden_dim, n_actions)
def forward(self, x):
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
return self.fc3(x)
n_states = 4
n_actions = 2
hidden_dim = 256
input_data = np.random.randn(1, 4).astype(np.float32)
pt_inputs = torch.from_numpy(input_data)
torch_model = MLP(n_states, n_actions, hidden_dim)
ov_model = ov.convert_model(torch_model, example_input=[input_data])
ov_compiled_model = ov.compile_model(ov_model, device_name="CPU")
with torch.no_grad():
torch_output = torch_model(pt_inputs).numpy()
ov_output = ov_compiled_model([input_data])[0]
np.testing.assert_allclose(ov_output, torch_output, atol=1e-4)
n = 10000
start = time.time()
with torch.no_grad():
for i in range(n):
torch_model(pt_inputs)
print(f"pt: {(time.time() - start) / n:.6f}")
start = time.time()
for i in range(n):
ov_compiled_model([input_data])
print(f"ov: {(time.time() - start) / n:.6f}") Returns:
Could you provide script to reproduce your issue? |
@mvafin Here is the script: https://github.com/qiwang067/openvino_rl/blob/main/dqn.py |
OpenVINO Version
2023.3.0
Operating System
Ubuntu 20.04 (LTS)
Device used for inference
None
OpenVINO installation
PyPi
Programming Language
Python
Hardware Architecture
x86 (64 bits)
Model used
MLP
Model quantization
No
Target Platform
No response
Performance issue description
Using 3-layer MLP to build a policy network for reinforcement learning with PyTorch, but inference slows down after conversion with OpenViNO:
3-layer MLP with PyTorch:
Step-by-step reproduction
Download and execute the following program:
https://github.com/qiwang067/openvino_rl/blob/main/dqn.py
Issue submission checklist
The text was updated successfully, but these errors were encountered: