AirSim and MATLAB integration using python problem

[hashikemu]

Hi, I'm Japanese student. and, i'm new to airsim,github,python, and matlab.
I want to use airsim for visual odometry. so, I made this matlab script.

clc
clear

%%addpath windows起動ごとにリセットされるっぽいのでここでチェックして、必要に応じて加える
if count(py.sys.path,'C:\Users\hashimoto\git\AirSim\PythonClient') == 0
    insert(py.sys.path,int32(0),'C:\Users\hashimoto\git\AirSim\PythonClient');
end


%%connect to the AirSim simulator 
client=py.AirSimClient.CarClient; %クライアントの宣言
client.confirmConnection();
client.enableApiControl(true); %接続とAPIの有効化
car_controls = py.AirSimClient.CarControls; %コントローラーの宣言


%%車両状態の取得と表示
car_state = client.getCarState();
state = car_state.kinematics_true.orientation.x_val

%%車両状態の指定
car_controls.throttle = 0.15
car_controls.steering = 1.0
client.setCarControls(car_controls)

%%画像の取得 badcastが出て止まる
response = client.simGetImages(py.list({    py.AirSimClient.ImageRequest(0, 0, false, false),    py.AirSimClient.ImageRequest(1, 0, false, false)    }))

client.reset()
client.enableApiControl(false)

this script uses matlab's python call function. this script requires follow this page's task. https://github.com/Microsoft/AirSim/blob/master/docs/python.md

but, in simGetImages function, exception error occurred. this is error message.

エラー: future>get (line 45)
Python エラー: RPCError: rpclib: function 'simGetImages' (called with 1 arg(s)) threw an exception. The
exception contained this information: bad cast.

エラー: session>call (line 41)
        return self.send_request(method, args).get()

エラー: AirSimClient>simGetImages (line 262)
        responses_raw = self.client.call('simGetImages', requests)

I can't determine whether "bad cast" error is by matlab or by msgpack.
so, can anyone give me advice?

[hashikemu]

Hello. About the above problem. I solved my self by writing the following program. I hope it will be helpful for linking MATLAB and Airsim. thanks.

%% 初期化
%初期化
clc
clear

%パス追加 windows起動ごとにリセットされるっぽいのでここでチェックして、必要に応じて加える
if count(py.sys.path,'C:\Users\hashimoto\git\AirSim\PythonClient') == 0
    insert(py.sys.path,int32(0),'C:\Users\hashimoto\git\AirSim\PythonClient');
end

%画像配列の確保
img_h=288;
img_w=512;
img_px=img_h*img_w;
resimg=zeros(img_w,img_h,3,'uint8'); %y*xのサイズになるらしい
resimg2=zeros(img_h,img_w,3,'uint8'); %転置先の配列

%figureの確保
f_vel=figure;
hold on;
f_xy=figure;
hold on;
f_cam=figure;

%キー入力受付用figure
%hf=figure('position',[0 0 eps eps],'menubar','none');


%% 接続
%connect to the AirSim simulator
client=py.AirSimClient.CarClient; %クライアントの宣言
client.confirmConnection();
client.enableApiControl(true); %接続とAPIの有効化
car_controls = py.AirSimClient.CarControls; %コントローラーの宣言



%% 制御ループ
client.simPause(logical(true))
while(1)
%状態更新
client.simContinueForTime(0.01) %10ms間隔で更新

%車両状態の取得と表示
car_state = client.getCarState();
state_t=car_state.timestamp;
state_vel = car_state.speed;
state_xp = car_state.kinematics_true.position.x_val;
state_yp = car_state.kinematics_true.position.y_val;
state_zp = car_state.kinematics_true.position.z_val;


%カラー画像の取得
response = cell(client.simGetImages(py.list({    py.AirSimClient.ImageRequest(int8(0), int8(0), logical(false), logical(false))    })));
imgdata=uint8(response{1,1}.image_data_uint8);
u=1;
for t = 1:img_h*img_w
    resimg(t)=          imgdata(u);
    resimg(t+img_px)=   imgdata(u+1);
    resimg(t+img_px*2)= imgdata(u+2);
    u=u+4;
end
resimg2=permute(resimg,[2 1 3]);


%車両状態の指定
car_controls.throttle = 0.30;
car_controls.steering = 1.0;

%入力の適用
client.setCarControls(car_controls);

%グラフ表示
figure(f_xy);
plot (state_xp,-state_yp,'o-');
daspect([1 1 1]);
figure(f_vel);
plot (state_t,state_vel,'o-');
figure(f_cam);
image(resimg2)
daspect([1 1 1]);

%pause(0.0001) %py.time.sleep(20)


end

client.simPause(logical(false))




%% 終了
client.reset()
client.enableApiControl(false)

[Junayed5259]

Hello hashikemu,

Were you successful to run airsim from matlab from the above code? This line 'client=py.AirSimClient.CarClient;' gives me error: Unable to resolve the name py.AirSimClient.CarClient. Can you please help me with that?

[Junayed5259]

Moreover, this link : https://github.com/Microsoft/AirSim/blob/master/docs/python.md is also not available anymore.

[hashikemu]

The last time I used it was 4 years ago, so the situation may have changed,
but it is necessary to build the environment according to the following contents.
https://jp.mathworks.com/help/matlab/matlab_external/install-supported-python-implementation.html?lang=en
https://github.com/Microsoft/AirSim/blob/master/PythonClient/car/hello_car.py

If the environment has already been built,
please check if the Python Client path can be added to the Python environment on MATLAB.
Please rewrite the path addition process as follows.

if count(py.sys.path,'path to the pythonclient in your environment') == 0
    insert(py.sys.path,int32(0),'path to the pythonclient in your environment');
end

The following pages are likely to replace the missing documents.
https://microsoft.github.io/AirSim/apis/