Squashed 'opendbc/' changes from c0eba096..bdcd31df

bdcd31df reducing precision
1263500c scale is supposed to be 0-255
7c4d34a shorter scale, shift aftert scale
7971a608 maybe scale was too long
be5b34b7 fingers crossed 0-1 is the desired range...
27368e96 accidental dup
79fe914e Added Pedal entries
63540dd2 add interceptor
a1aa3b78 Cleanup of Nissan DBC (commaai#218)
c1f76482 reverting changes to Chrysler: speed message seems different from car to car
2de8e9e8 Chrysler: Speed msg is 5 bytes
f4ac315e Chrysler: add counter to 514
0fb62cf8 Toyota Blind Spot Monitor (TSS2-only?) (commaai#219)
2ae0327e GitHub actions (commaai#217)
fbbba94a GM: use common gear VALs
dacd8b46 Chrysler commonize gear VALs
d338b9fb Add support for 2019 Civic Sedan Diesel. Split GAS_PEDAL_2 out to each car. (commaai#215)
d68f8ff0 Clean up .gitignore
e82757b9 Add Lexus NX300H (commaai#214)
745a2bb0 Removed non high voltage items. Added units.
fafd091a Added High Voltage Management to powertrain file.
15b6b316 Update subaru_global_2017.dbc
a12af19a Add steering error signal in subaru global
d411c175 correct max value
881c065d add electrical gear and fix driver torque
1bbac4b8 Add reference to CANdevStudio in README file
8f70ea01 Adds 2016 Honda CR-V Executive
5081966f One more fix
fa5dc680 Fix honda dbc files after steer torque addition
e4dfb2fa update honda steering signals (commaai#208)
53fc4487 added generator test (commaai#207)
cb27d6e3 Honda Nidec: add new ACC_HUD signals to all other cars other than the CIVIC
6e6779f9 build on mac AND linux, also gotta not use these python files
79dbfc1a build on mac
4f82d01e gitignore
5cb83454 Honda FCM: diagnostic signals
d309cdce Added linter to opendbc (commaai#203)
d452706f add requirements.txt
ec3b4595 deterministic dependency order
a265d351 Azure pipelines ci (commaai#202)
bce9a2e1 packer depends on libdbc
5d5fdd6a no more python version of libdbc, everything through cython
541705bf move CANDefine to parser code
da25c52a add test for can define
0ba7926b unify can packer and parser
25d88009 consistent naming
a5c640a5 fix linter
be210fef remove obsolete make file
ffd9dca7 opendbc needs cereal
b559f63d remove more make
d0929496 seems to work now
41e80836 don't make
3254d1fc think scons works
eb78f6aa scons sort of working
0ef1e35d fix gitignore
e155e017 Can migration (commaai#199)
3eded83a Honda: correct steering torque sensor sign to be consistent with standard convention (left+)
32f70e2f Fix outback endianness consistency (commaai#196)
a7da471f Update subaru_outback_2015_eyesight.dbc (commaai#195)

git-subtree-dir: opendbc
git-subtree-split: bdcd31dff8e38467281e1b0f865abe978b04d990

.github/workflows/tests.yml

@@ -0,0 +1,24 @@
+name: Tests
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-16.04
+
+    steps:
+      - uses: actions/checkout@v2
+      - run: |
+          set -e
+          docker build -t opendbc .
+        name: "Build"
+      - run: |
+          docker run opendbc bash -c "python -m unittest discover opendbc"
+        name: "Unit tests"
+      - run: |
+          docker run opendbc bash -c "cd opendbc/can/tests/linter_python; PYTHONPATH=/ ./flake8_opendbc.sh"
+          docker run opendbc bash -c "cd opendbc/can/tests/linter_python; PYTHONPATH=/ ./pylint_opendbc.sh"
+        name: "Python linter"
+      - run: |
+          docker run opendbc bash -c "cd opendbc/can/tests/; PYTHONPATH=/ ./test_generator.sh"
+        name: "Generator test"

.gitignore

@@ -1,2 +1,13 @@
 *.pyc
+*.os
+*.tmp
+*.dylib
 .*.swp
+.DS_Store
+
+can/*.so
+can/build/
+can/obj/
+can/packer_pyx.cpp
+can/parser_pyx.cpp
+can/packer_impl.cpp

Dockerfile

@@ -0,0 +1,25 @@
+from ubuntu:16.04
+
+RUN apt-get update && apt-get install -y libzmq3-dev clang wget git autoconf libtool curl make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl
+
+RUN curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
+ENV PATH="/root/.pyenv/bin:/root/.pyenv/shims:${PATH}"
+RUN pyenv install 3.7.3
+RUN pyenv global 3.7.3
+RUN pyenv rehash
+
+COPY requirements.txt /tmp/
+RUN pip install -r /tmp/requirements.txt
+
+ENV PYTHONPATH=/project
+
+# TODO: Add tag to cereal
+RUN git clone https://github.com/commaai/cereal.git /project/cereal
+RUN /project/cereal/install_capnp.sh
+
+WORKDIR /project
+
+COPY SConstruct .
+COPY . /project/opendbc
+
+RUN scons -c && scons -j$(nproc)

README.md

@@ -16,6 +16,9 @@ Wondering what's the DBC file format? [Here](http://www.socialledge.com/sjsu/ind
 
 Use [panda](https://github.com/commaai/panda) to connect your car to a computer.
 
+### How to use reverse engineered DBC
+To create custom CAN simulations or send reverse engineered signals back to the car you can use [CANdevStudio](https://github.com/GENIVI/CANdevStudio) project.
+
 ### DBC file preprocessor
 
 DBC files for different models of the same brand have a lot of overlap. Therefore, we wrote a preprocessor to create DBC files from a brand DBC file and a model specific DBC file. The source DBC files can be found in the generator folder. After changing one of the files run the generator.py script to regenerate the output files. These output files will be placed in the root of the opendbc repository and are suffixed by _generated.
@@ -42,7 +45,7 @@ For example:
     SG_ VEHICLE_SPEED : 7|15@0+ (0.01,0) [0|250] "kph" PCM
     ```
 
-- Signal's size: always use the smallest amount of bits possible. For example, let's say I'm reverse engineering the gas pedal position and I've determined that it's in a 3 bytes message. For 0% pedal position I read a message value of `0x00 0x00 0x00`, while for 100% of pedal position I read `0x64 0x00 0x00`: clearly, the gas pedal position is within the first byte of the message and I might be tempted to define the signal `GAS_POS` as:
+- Signal size: always use the smallest amount of bits possible. For example, let's say I'm reverse engineering the gas pedal position and I've determined that it's in a 3 bytes message. For 0% pedal position I read a message value of `0x00 0x00 0x00`, while for 100% of pedal position I read `0x64 0x00 0x00`: clearly, the gas pedal position is within the first byte of the message and I might be tempted to define the signal `GAS_POS` as:
     ```
     SG_ GAS_POS : 7|8@0+ (1,0) [0|100] "%" PCM
     ```

SConstruct

@@ -0,0 +1,57 @@
+import os
+import subprocess
+
+zmq = 'zmq'
+arch = subprocess.check_output(["uname", "-m"], encoding='utf8').rstrip()
+
+cereal_dir = Dir('.')
+
+cpppath = [
+    '#',
+    '#cereal',
+    "#cereal/messaging",
+    "#opendbc/can",
+    '/usr/lib/include',
+]
+
+AddOption('--test',
+          action='store_true',
+          help='build test files')
+
+AddOption('--asan',
+          action='store_true',
+          help='turn on ASAN')
+
+ccflags_asan = ["-fsanitize=address", "-fno-omit-frame-pointer"] if GetOption('asan') else []
+ldflags_asan = ["-fsanitize=address"] if GetOption('asan') else []
+
+env = Environment(
+  ENV=os.environ,
+  CC='clang',
+  CXX='clang++',
+  CCFLAGS=[
+    "-g",
+    "-fPIC",
+    "-O2",
+    "-Werror=implicit-function-declaration",
+    "-Werror=incompatible-pointer-types",
+    "-Werror=int-conversion",
+    "-Werror=return-type",
+    "-Werror=format-extra-args",
+  ] + ccflags_asan,
+  LDFLAGS=ldflags_asan,
+  LINKFLAGS=ldflags_asan,
+
+  CFLAGS="-std=gnu11",
+  CXXFLAGS="-std=c++14",
+  CPPPATH=cpppath,
+)
+
+Export('env', 'zmq', 'arch')
+
+cereal = [File('#cereal/libcereal.a')]
+messaging = [File('#cereal/libmessaging.a')]
+Export('cereal', 'messaging')
+
+SConscript(['cereal/SConscript'])
+SConscript(['opendbc/can/SConscript'])

acura_ilx_2016_can_generated.dbc

@@ -86,6 +86,13 @@ BO_ 420 VSA_STATUS: 8 VSA
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" EON
  SG_ CHECKSUM : 59|4@0+ (1,0) [0|15] "" EON
 
+BO_ 427 STEER_MOTOR_TORQUE: 3 EPS
+ SG_ CONFIG_VALID : 7|1@0+ (1,0) [0|1] "" EON
+ SG_ MOTOR_TORQUE : 1|10@0+ (1,0) [0|256] "" EON
+ SG_ OUTPUT_DISABLED : 22|1@0+ (1,0) [0|1] "" EON
+ SG_ COUNTER : 21|2@0+ (1,0) [0|3] "" EON
+ SG_ CHECKSUM : 19|4@0+ (1,0) [0|15] "" EON
+
 BO_ 432 STANDSTILL: 7 VSA
  SG_ CONTROLLED_STANDSTILL : 0|1@0+ (1,0) [0|1] "" EON
  SG_ WHEELS_MOVING : 12|1@0+ (1,0) [0|1] "" EON
@@ -164,7 +171,7 @@ BO_ 780 ACC_HUD: 8 ADAS
  SG_ BOH : 38|1@0+ (1,0) [0|1] "" BDY
  SG_ ACC_PROBLEM : 37|1@0+ (1,0) [0|1] "" BDY
  SG_ FCM_OFF : 36|1@0+ (1,0) [0|1] "" BDY
- SG_ BOH_2 : 35|1@0+ (1,0) [0|1] "" BDY
+ SG_ FCM_OFF_2 : 35|1@0+ (1,0) [0|1] "" BDY
  SG_ FCM_PROBLEM : 34|1@0+ (1,0) [0|1] "" BDY
  SG_ RADAR_OBSTRUCTED : 33|1@0+ (1,0) [0|1] "" BDY
  SG_ ENABLE_MINI_CAR : 32|1@0+ (1,0) [0|1] "" BDY
@@ -174,10 +181,12 @@ BO_ 780 ACC_HUD: 8 ADAS
  SG_ BOH_4 : 42|1@0+ (1,0) [0|3] "" BDY
  SG_ BOH_5 : 41|1@0+ (1,0) [0|3] "" BDY
  SG_ CRUISE_CONTROL_LABEL : 40|1@0+ (1,0) [0|3] "" BDY
- SG_ HUD_DISTANCE_3 : 52|1@0+ (1,0) [0|1] "" BDY
- SG_ IMPERIAL_UNIT : 54|1@0+ (1,0) [0|1] "" BDY
  SG_ SET_ME_X01_2 : 55|1@0+ (1,0) [0|1] "" BDY
+ SG_ IMPERIAL_UNIT : 54|1@0+ (1,0) [0|1] "" BDY
+ SG_ HUD_DISTANCE_3 : 52|1@0+ (1,0) [0|1] "" BDY
+ SG_ CHIME : 51|3@0+ (1,0) [0|1] "" BDY
  SG_ SET_ME_X01 : 48|1@0+ (1,0) [0|1] "" BDY
+ SG_ ICONS : 63|2@0+ (1,0) [0|1] "" BDY
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" BDY
  SG_ CHECKSUM : 59|4@0+ (1,0) [0|3] "" BDY
 
@@ -271,10 +280,11 @@ BO_ 342 STEERING_SENSORS: 6 EPS
  SG_ CHECKSUM : 43|4@0+ (1,0) [0|3] "" EON
 
 BO_ 399 STEER_STATUS: 7 EPS
- SG_ STEER_TORQUE_SENSOR : 7|16@0- (1,0) [-31000|31000] "tbd" EON
- SG_ STEER_TORQUE_MOTOR : 23|16@0- (1,0) [-31000|31000] "tbd" EON
+ SG_ STEER_TORQUE_SENSOR : 7|16@0- (-1,0) [-31000|31000] "tbd" EON
+ SG_ STEER_ANGLE_RATE : 23|16@0- (-0.1,0) [-31000|31000] "deg/s" EON
  SG_ STEER_STATUS : 39|4@0+ (1,0) [0|15] "" EON
  SG_ STEER_CONTROL_ACTIVE : 35|1@0+ (1,0) [0|1] "" EON
+ SG_ STEER_CONFIG_INDEX : 43|4@0+ (1,0) [0|15] "" EON
  SG_ COUNTER : 53|2@0+ (1,0) [0|3] "" EON
  SG_ CHECKSUM : 51|4@0+ (1,0) [0|3] "" EON
 

acura_rdx_2018_can_generated.dbc

@@ -86,6 +86,13 @@ BO_ 420 VSA_STATUS: 8 VSA
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" EON
  SG_ CHECKSUM : 59|4@0+ (1,0) [0|15] "" EON
 
+BO_ 427 STEER_MOTOR_TORQUE: 3 EPS
+ SG_ CONFIG_VALID : 7|1@0+ (1,0) [0|1] "" EON
+ SG_ MOTOR_TORQUE : 1|10@0+ (1,0) [0|256] "" EON
+ SG_ OUTPUT_DISABLED : 22|1@0+ (1,0) [0|1] "" EON
+ SG_ COUNTER : 21|2@0+ (1,0) [0|3] "" EON
+ SG_ CHECKSUM : 19|4@0+ (1,0) [0|15] "" EON
+
 BO_ 432 STANDSTILL: 7 VSA
  SG_ CONTROLLED_STANDSTILL : 0|1@0+ (1,0) [0|1] "" EON
  SG_ WHEELS_MOVING : 12|1@0+ (1,0) [0|1] "" EON
@@ -164,7 +171,7 @@ BO_ 780 ACC_HUD: 8 ADAS
  SG_ BOH : 38|1@0+ (1,0) [0|1] "" BDY
  SG_ ACC_PROBLEM : 37|1@0+ (1,0) [0|1] "" BDY
  SG_ FCM_OFF : 36|1@0+ (1,0) [0|1] "" BDY
- SG_ BOH_2 : 35|1@0+ (1,0) [0|1] "" BDY
+ SG_ FCM_OFF_2 : 35|1@0+ (1,0) [0|1] "" BDY
  SG_ FCM_PROBLEM : 34|1@0+ (1,0) [0|1] "" BDY
  SG_ RADAR_OBSTRUCTED : 33|1@0+ (1,0) [0|1] "" BDY
  SG_ ENABLE_MINI_CAR : 32|1@0+ (1,0) [0|1] "" BDY
@@ -174,10 +181,12 @@ BO_ 780 ACC_HUD: 8 ADAS
  SG_ BOH_4 : 42|1@0+ (1,0) [0|3] "" BDY
  SG_ BOH_5 : 41|1@0+ (1,0) [0|3] "" BDY
  SG_ CRUISE_CONTROL_LABEL : 40|1@0+ (1,0) [0|3] "" BDY
- SG_ HUD_DISTANCE_3 : 52|1@0+ (1,0) [0|1] "" BDY
- SG_ IMPERIAL_UNIT : 54|1@0+ (1,0) [0|1] "" BDY
  SG_ SET_ME_X01_2 : 55|1@0+ (1,0) [0|1] "" BDY
+ SG_ IMPERIAL_UNIT : 54|1@0+ (1,0) [0|1] "" BDY
+ SG_ HUD_DISTANCE_3 : 52|1@0+ (1,0) [0|1] "" BDY
+ SG_ CHIME : 51|3@0+ (1,0) [0|1] "" BDY
  SG_ SET_ME_X01 : 48|1@0+ (1,0) [0|1] "" BDY
+ SG_ ICONS : 63|2@0+ (1,0) [0|1] "" BDY
  SG_ COUNTER : 61|2@0+ (1,0) [0|3] "" BDY
  SG_ CHECKSUM : 59|4@0+ (1,0) [0|3] "" BDY
 
@@ -259,8 +268,8 @@ BO_ 392 GEARBOX: 6 XXX
  SG_ GEAR : 36|5@0+ (1,0) [0|31] "" EON
 
 BO_ 399 STEER_STATUS: 6 EPS
- SG_ STEER_TORQUE_SENSOR : 7|12@0- (1,0) [-2047.5|2047.5] "tbd" EON
- SG_ STEER_TORQUE_MOTOR : 23|16@0- (1,0) [-31000|31000] "tbd" EON
+ SG_ STEER_TORQUE_SENSOR : 7|12@0- (-1,0) [-2047.5|2047.5] "tbd" EON
+ SG_ STEER_ANGLE_RATE : 23|16@0- (-0.1,0) [-31000|31000] "deg/s" EON
  SG_ STEER_STATUS : 35|4@0+ (1,0) [0|15] "" EON
  SG_ STEER_CONTROL_ACTIVE : 36|1@0+ (1,0) [0|1] "" EON
  SG_ COUNTER : 45|2@0+ (1,0) [0|3] "" EON

can/SConscript

@@ -0,0 +1,27 @@
+Import('env', 'cereal')
+
+import os
+from opendbc.can.process_dbc import process
+
+dbcs = []
+for x in sorted(os.listdir('../')):
+  if x.endswith(".dbc"):
+    def compile_dbc(target, source, env):
+      process(source[0].path, target[0].path)
+    in_fn = [os.path.join('../', x), 'dbc_template.cc']
+    out_fn = os.path.join('dbc_out', x.replace(".dbc", ".cc"))
+    dbc = env.Command(out_fn, in_fn, compile_dbc)
+    dbcs.append(dbc)
+
+
+libdbc = env.SharedLibrary('libdbc', ["dbc.cc", "parser.cc", "packer.cc", "common.cc"]+dbcs, LIBS=["capnp", "kj"])
+
+# packer
+env.Command(['packer_pyx.so'],
+  [libdbc, 'packer_pyx.pyx', 'packer_pyx_setup.py'],
+  "cd opendbc/can && python3 packer_pyx_setup.py build_ext --inplace")
+
+# parser
+env.Command(['parser_pyx.so'],
+  [libdbc, cereal, 'parser_pyx_setup.py', 'parser_pyx.pyx', 'common.pxd'],
+  "cd opendbc/can && python3 parser_pyx_setup.py build_ext --inplace")

can/can_define.py

@@ -0,0 +1,2 @@
+from opendbc.can.parser_pyx import CANDefine # pylint: disable=no-name-in-module, import-error
+assert CANDefine