New OS for RPI4/5 causing Problems

[decodeais]

I wanted to set up RasperiPy4 again, as I use it as a true multifunctional device (Print spooler, web server, etc.).
However, with the latest version of the OS (Bookworm), some issues arose.

• Activating the VNC server is no longer as straightforward. "raspi-config" must be called , and then "WayVNC" needs to be switched to the X backend. After that, RealVNC can be activated as usual by enabling the "VNC" option in the "Interface Options". Additionally, the "wayvnc" backend had to be uninstalled, as it caused annoying delays during boot.
  The old instruction in the manual is no longer entirely accurate and could be confusing.

• Whenever "pip3" commands were executed in the installation script, the option "--break-system-packages" had to be added.

• Now that the new Python Version 3.11 is installed, the WHL for wxPython no longer fits. The reference in the manual to MarcDobler's page was useless as the link was dead (it should be removed from the manual).

Nevertheless, I managed to generate a suitable one:

sudo apt-get install libgtk-3-dev
sudo apt-get install libpng-dev libjpeg-dev libtiff-dev
pip install -U --no-binary :all: wxPython

The generated file "wxPython-4.2.1-cp311-cp311-linux_aarch64.whl" works,
The but the scales on the performance and speed indicators have disappeared.

These were the main issues. Initially, I would appreciate a suggestion on where to search for the missing scales. However, this is only a "Design" problem.

[WouterJD]

Yeah Yeah; "Python is a WORA (Write Once, Run Anywhere) language."; everybody may judge how true the statement is.
Sorry for the inconvenience, anyway.

Please suggest what changes help you forward, apart from the missing whl.

[WouterJD]

• The reference in the manual to MarcDobler's page was useless as the link was dead (it should be removed from the manual).

Hmmm... Did a quick check but don't find; please what page?

[decodeais]

Sorry, the address of marcDobler.com appeared in the message from the instalation script "4_InstallWxPython.sh".

[decodeais]

The missing scales look a little strange, and i would like to know, wether the generated whl in issue #440 has the same lost. It has the same wx revision number.

[WouterJD]

I do not have an RPI4/5 so cannot test, but perhaps others can...

[decodeais]

Will you add the missing Parameter "--break-system-packages"? Or will this brake the installation for the others?

[WouterJD]

I'm ok to do so, but in what line please

[decodeais]

every pip3 command needs it.

[decodeais]

I see two ways:

1. May be, thats it is easier to use the virtual envirement to determine a special Python Version. This would fix the error messages from the Script.
2. The second idea would be, somebody makes a minimum installation and compress it to the minimum image size (~3GB).
   One for Raspi0 and Raspi3/4/5.

[decodeais]

There is a strange behavior when I close the program. the terminal windows stays open and does not close.

22:31:31,016: FortiusAnt applies the MotorBrake power curve
22:31:35,484: Tacx2Dongle; Pedal Stroke Analysis active
22:31:35,484: ---------- BLE-devices are activated ----------
22:31:37,486: ---------- BLE-devices are deactivated ----------

Does this effect only appear with "bookworm"?
I think I will look into the source code. May be you can give me a hint.

[WouterJD]

It happens on Windows as well.
Not reproducible.

Perhaps a coding error waiting for threads to complete. I don't know

[decodeais]

For me the waiting the thread problem seems be solved. I could not find the reason. I fixed the problem by "brutal force". I look for the process id from main. At the end I kill it.
In FortiusAnt.py (731 )I get the PID:

#-----------------------------------------------------------------------------------
# Main program; when restart is required due to new parameters we will loop
#-----------------------------------------------------------------------------------
if __name__ == "__main__":
    multiprocessing.freeze_support()
    global RestartApplication, clv
    Kill.pid = os.getpid()
    RestartApplication = False

In FortiusAntGui.py (1502) I kill it:

if self.CloseButtonPressed == True:
            self.CloseButtonPressed = False     # Otherwise self.Close() is blocked
            self.Close()
            os.kill(Kill.pid, 9)

What do you think, will it have bad side effects?

[decodeais]

I found the reason for the disappeared Scales ( it was a white eagle on white ground problem). I found the reason in the speedmeter library. When I added the typecast to the color changing variables the blue background came back and the white scaling appeared again.
(speedmeter.py, line 843)

                #currCol = (r1 + rf, g1 + gf, b1 + bf)
                currCol = (r1 + int(rf), g1 +int(gf), b1 +int(bf))
                
                dc.SetBrush(wx.Brush(currCol))

I have no experience how to handle this kind of problem.
Should somebody write an issue on github for speedmeter.py ?

[WouterJD]

Hi @decodeais good work.

You have greatly re-engineered to the location where the problem is created, let's see how to help.

Some questions (I assume you can answer):

• In what function, the "currCol=" statement appears,
• the r1, rf, g1, gf and b1, bf may be provided by FortiusAnt; can you tell me in what function-call?

If you tell me this, it avoids work for myself.
I never downloaded the speedmeter code untill now.

[WouterJD]

os.kill(Kill.pid, 9)

I will look into it; in fact it suggests there is some flow-error and it circumvents the problem creatively; but still no beauty.
Kind a busy these days, but I will pick up.

[decodeais]

I tried to make a pull request for the speedometer software( I have no expearence with this things) . I found that there are other people withe the same problems with the intern typecast errors on the other instrumentts, but nothing seems to happen.
Please keep relaxed, for me it works. I dont't want to make stress, I want to help.

[WouterJD]

I assume it can be considered a calling error, do if you help me forward, we can solve in our code

[decodeais]

It is an internal problem in the speedmeter software. The only workaround you can do, is to change the color of the scale from white to black. it looks much better than the white on white.

[WouterJD]

Ok, please suggest code change

[decodeais]

1. For the Workaround:
   In FortiusAntGUI.py you can change the SetTicksColour from White to Black. But it looks not good.
2. More elegant:
   (speedmeter.py, line 843)

               #currCol = (r1 + rf, g1 + gf, b1 + bf)
               currCol = (r1 + int(rf), g1 +int(gf), b1 +int(bf))
               dc.SetBrush(wx.Brush(currCol))

[WouterJD]

2.. More elegant:

I cannot change this code (other than locally) so I assume I cannot help.

[decodeais]

I made my first pull request (I never done before). The problem was accepted and verified. I hope they will change it as soon as possible.
In my repository is a patched version. If you no how to install it from there, you can test it. When I checked the problem, I searched the file on my hard disk and patched the line.
But do you know that this is not only a Raspbian problem?
It affects all OS. The problem comes from the changes in Python3.10 upwards. The behavior between integer an float in calculations has changed.

[WouterJD]

The behavior between integer an float in calculations has changed.

I my opinion a fundamental design of Python error not to implement strict data typing...

[decodeais]

You don't have to declare the Type, but you have to find out to what it has mutated.
I agree.

[decodeais]

I'am afraid that this problem caused some displacements in the GUI. I had to move some Buttons.
You should think about fixing the Python version.
On the Raspberry it would be possible to fix the OS too.
I told you, I would prefer to make a image. We Would have to define:
OS version ,Preinstalled apps, Passwords ... Then we could make a installation and compress it like the Kodi or Elec images.
The last problem would be to find a Server, but I think we don`t need much space.

[WouterJD]

Well, not helping the subject but sure fun to read https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

I did not get to the end of this stunning text 😉

[decodeais]

This is what it looks like when a mathematician tries to code

[decodeais]

It was a hard work, so I hope it realy works.
When I used these Wheels thy worked. But I know there are sometimes side effects tat it is only an illusion.

This is my default for <3.10:
wxPython-4.2.0-cp39-cp39-linux_armv7l.whl

This was my first test for Raspbian Bookworm with Python3.11, it has still the Speedmeter bug:
wxPython-4.2.1-cp311-cp311-linux_aarch64.whl

This is unofficial pre release with bug fix:
wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl

Here are the wheels for normal Linux with Python >3.09
wxPython-4.2.2a1-cp310-cp310-linux_x86_64.whl
wxPython-4.2.2a1-cp311-cp311-linux_x86_64.whl

I hope somebody can use it and tell me if there is something wrong.

Here is the script i used to generate these wheels. It has to be tested on some fresh systems. Nevertheless its a good starting point for own generated wheels:

#!/bin/bash
# export PATH="$HOME/.local/bin:$PATH"

export DOXYGEN=/usr/bin/doxygen # for Raspian we must use the internal dxygen
# Update package lists
sudo apt-get update
sudo apt install doxygen
# Install required packagesexport PATH="$HOME/.local/bin:$PATH"
sudo apt install -y dpkg-dev build-essential python3-dev freeglut3-dev libgl1-mesa-dev \
libglu1-mesa-dev libgstreamer-plugins-base1.0-dev libgtk-3-dev libjpeg-dev libnotify-dev \
libpng-dev libsdl2-dev libsm-dev libtiff-dev libwebkit2gtk-4.0-dev libxtst-dev \
libgstreamer1.0-dev libjpeg-dev libtiff-dev libsm-dev

# Create directory for building
mkdir -p ~/WheelBuild311

# Change directory to the build directory
cd ~/WheelBuild311

# Create and activate virtual environment
python3.11 -m venv WheelBuilder311_py
source ~/WheelBuild311/WheelBuilder311_py/bin/activate

# Upgrade pip and install required Python packages
pip install -U pip six wheel setuptools sip requests

# Clone Phoenix repository
git clone https://github.com/wxWidgets/Phoenix.git --branch master
cd Phoenix
git submodule init
git submodule update
git submodule update --init --recursive

# Build Phoenix
python build.py dox etg --nodoc sip build bdist_wheel
#python build.py build bdist_wheel

May be it would be necessary to use a virtual machine for linux. A fresh Raspbian should not be a problem.

[WouterJD]

But where can we save such wheels? It would be stupid to compile it allways new.

---

From the beginning - I was 100% new to python, github and visual studio code - I created the folder structure, including the .exe and .pdf files. Later I learned, github is intended for source-files only and large deliverables should be stored elsewhere; not that github blocks in any way.

The .exe / .pdf files litter the repository (all previous versions are still available) but I see no straight-forward solution.

If anyone reads this and has suggestions - feel free to suggest

---

For the files you have created, the following approach is good for me:

• We write a support page in the wiki with the links
• Store the files in the Supportfiles directory.

If you write a pull request with the wheels at the suggested location, we can proceed like that and keep the files available for others. Please give the files a clear name...

---

And again:
If anyone reads this and has suggestions - feel free to suggest

[decodeais]

First you can copy the link, I let them there. Everybody who has this link can use the file.
I the link in your installation script is of same kind, I think.

[decodeais]

I think the reason for the problem is that the chief from this project was unable to work for this problem for two years. They restarted the project, made the changes for python 3.11, the tool to generate the wheels does not work for linux and windows at this moment. This will change, I hope.

[WouterJD]

First you can copy the link, I let them there. Everybody who has this link can use the file. I the link in your installation script is of same kind, I think.

Fair enough, easy stuff: https://github.com/WouterJD/FortiusANT/wiki#special-files,

Accessible for everybody who needs.

[decodeais]

Please be strong i have some more question:

• I always wondering why pint the cadence meter a darker blue as
  background color, shouldn't it be "colorTacxFortius".

self.Revs.SetFirstGradientColour(wx.BLUE)

• Did you mention that Linux changed from X11 to Wayland? In my first test I switched the desktop system to X11. The reason was that VNC only works with X11 on Rasbian. On X11 I have seen only little displacements of the buttons. With Wayland the behavier is worse. The meters are cut by the window ( left and bottom) and the gear window is complete confused (stacked). Do you have an idea what is going wrong, wher do you calculate the size of the window?
• The install of 3_InstallNodeJs.sh fails, do we really need it?

The good news:

• The wheel works in a fresh installation too. I changed the script for
  Python3.11:

wxPYTHON=wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl
GDRIVE=https://drive.google.com/file/d/1pPrAQLd3plFiuZYifh1HSn_Kr5Iy2EVr/view?usp=sharing
gdown --id 1pPrAQLd3plFiuZYifh1HSn_Kr5Iy2EVr --output $wxPYTHON

• I think I understand the virtual envirement a little ( I learned it fighting with the wheels).
  It is a good way to solve the pip installation problem with the --break-system-packages option.

[decodeais]

I fixed the size problem for Wayland desktop, with the addition of a constant value to the size. I know its not the right way, but it is dificult to find out your strategy of your window size calculation.

self.SetSize (BitmapX + BitmapW + 40, BitmapY + BitmapH+60)
self.panel.SetSize (BitmapX + BitmapW + 40, BitmapY + BitmapH+60)

For the gearbox, I had to change the size too.

frameX = 60      #2 * Margin +     pCranckset.Size[0] + 15
frameY = 200    #3 * Margin + 2 * pCranckset.Size[1] + 39
 

Now it looks quite normal again.

[decodeais]

Next Problem but fixed. The old autostart can not work. These are working:
Autostart_On.sh

#!/bin/bash 
echo "[autostart]" >> ~/.config/wayfire.ini
echo "a0 = lxterminal --working-directory=/home/pi/FortiusANT/raspberry -e \$HOME/FortiusANT/raspberry/FortiusAnt.sh" >> ~/.config/wayfire.ini

Autostart_Off.sh

#!/bin/bash 
sed -i '/\[autostart\]/,/^$/d' ~/.config/wayfire.ini

[decodeais]

I have put my changes to github: https://github.com/decodeais/FortiusANT/tree/Wayland
I changed the installation to virtual enviroment for Python. Every thing for FortiusAnts Python is hosted in the FortiusAnt directory. I if the Python version of the Raspi changes,the version for FortiusAnt stays 3.11 .
The installation can be started by this script:
./FortiusANT/raspberry/FortiusAntInstall.sh
When its ready you can start it in "Others" of the Raspi startmenue.
I think I have fixed all problems for me. Next days I will test a fresh installation.

[WouterJD]

but it is dificult to find out your strategy of your window size calculation.

I understand that. I think it's been experimental what looks goed.
The +40 and +60 have to with the border sizes, which I did not find anywhere. The constants cause all controls to be same/related shape.

[decodeais]

I took this image 1.1GB Raspbian Bookworm and installed it with my installation script.
It worked. I hope somebody can check it.
My problem is that the first video output from my Raspi is broken and the Raspi gets incredible hot.
I will have to buy a new one.

[zapiens]

Hi, I can check it out. I just got a RP4 and am going to try to set it up.

`Installing collected packages: six, numpy, wxPython

Successfully installed numpy-1.26.4 six-1.16.0 wxPython-4.2.2a1
.........
Reading state information... Done
E: Unable to locate package wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl
E: Couldn't find any package by glob 'wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl'
E: Couldn't find any package by regex 'wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl'
bash: stop.sh: No such file or directory
`

[decodeais]

It looks for me that it can not find the stop.sh script. I get this error messages too.
When I tested it this morning renamed the FortiusAnt Directory to have a backup. When i started the new installation it stopped at the same position but the reason was a full sd-card. I erased the backup and then everything worked ( i got the same error message but "no stop.sh not found").
if you want to check the wxPython installation :

#!/bin/bash
cd /home/pi/FortiusANT/raspberry
source FortAntEnv/bin/activate #start the virtual envirement
pip list
./stop.sh

Please check the execute permission for stop.sh, it was lost on my Raspi.
Remember all Python installations are only valid when you started the virtual enviroment in the terminal:

#!/bin/bash
cd /home/pi/FortiusANT/raspberry
source FortAntEnv/bin/activate #start the virtual enviroment

[decodeais]

I found the reason for the error messages there where 2 installations in the script. One from the download and one stock install with apt-get. The "stop.sh" has now its permission to run.

[zapiens]

I will do a complete new install from a clean sd card.

Do you have have your install script on the Wayland branch?

I noted you mix /home/pi with $HOME the latter is preferable I think as not to force people having the pi user.

[decodeais]

I know, but I have changed so much in this Wayland Branch. My aim is to get a good installation as soon as possible.
Next step for me will be to shrink the complete
Sd-card and put it into an image ready to use like the media player images. Compressing last very long, so I must be sure.
You can start the script with the parameter n. Then the breaks between the installation blocks should be disabled. Only the stops from the apt-get commands will come, at the first time.

[zapiens]

I saw you made a Zapiens commit. I branched off at that commit to make the changes I thought of but I could not push. I forked the repo instead to try a complete hands free installation.

[decodeais]

If you succeed I would like to test yours. I think it would be a good idea to give the apt-get commands a yes parameter. Then it would run to the end in connection with install script n parameter. If it stopped with an error I allways tried to edit the script and restaerted it, fresh installs only to test.
I have always problems to understand Git. Chatgpt tries to help me but everything is still confusing.
If you think it works you could send me a "pull request", if I understood it right.
The next days i will receive my new Raspi, then I will be able to make more tests.

[zapiens]

Many iterations when the aim is totally hands free, always a little thing left to do. I am almost there ..... ha ha

If you have any questions about git feel free to ask me.

Finally no hands and it is up and running on an RP4

[decodeais]

This sounds good!
I was a little exhausted from building the wxPython wheel last week. I think we should place the script for the wheel somewhere in the "raspery" directory. It was always a problem in the past to get the right one.

I will get the new Raspi next days.we I orderd a Zero 2W too, so i can check this one too.
We should check the instalation on linux too. I think the only thing we must change is th wxPython wheel, but i have allready one generated.

The old autostart, setup onnly works only on X11 on Wayland we can use my autostart_ON/OFF script.
I hope we don't need nodejs installation. It does not work on bookworm. I have a alternative way in the bullseye installation script, but it was always a little mystique.

The little help scripts will make problems when somebody forgets to start the virtual environment first, we should think about how to handle.

Now the most important for me, how to get the last version from you to merge ( I am a github analphabet).
When I look into your repository i only find an old version. The latest version I get only through the timeline.
Did you forget somthing to publish, or can you make a pull request?
I hope you know it.

[zapiens]

Good work in getting wxPython to build.

If you do this:

git remote add zapiens https://github.com/zapiens/FortiusANT.git
git fetch zapiens

You will have mine fork together with yours. Nothing of what you have done is affected.

git log zapiens/dag/issue_457

Shows my commits, now only one as I squashed them together Note the hash of my commit: 6ceca98

You can insert that commit where you see fit by:

git cherry-pick 6ceca9810

I think pull requests are an overkill at this stage with work in progress.

[decodeais]

I have got your changes. I lost orientatio for a while between all the repositories, branches..
I hope you get no nonsense requests from my repostory.
Is your connection problem solved(Waiting for response from Bluetooth device)?

[zapiens]

I hope you one day will see the beauty of Merkle trees and the git confusion will be gone ;)

I still have the connection problem. I have tried to pair with my Android phone, windows 11, Ubuntu 22.04 all with the same outcome. I don't know how to debug this or what is expected at this point.

In windows the 'FortiusAntTrainer' is shown as an Audio device and fails to pair. In Ubuntu the pairing comes up with the numbers that should match. Clicking confirm on both computers the Raspberry has the dialogue box 'Waiting for response from Bluetooth device' In ubuntu I get the error:

[  394.175497] Bluetooth: hci0: Opcode 0x 401 failed: -16

in dmesg

[zapiens]

I am totally lost on the bluetooth. I can connect now connect the raspberry to ubuntu 22.04/windows 11 and send files to/from ubuntu and only from windows 11.

In windows 11 the fortius appears as an audio device.

In neither ubuntu or windows the FortiusAntTrainer appears in training software. So far I have only tested with Zwift and GoldenCheetah.

In ubuntu the device looks like:

D8:3A:DD:36:A4:46
public
FortiusAntTrainer
FortiusAntTrainer
0x7c0000
0x0000
blueman
yes
yes
no
no
-50 dBm (0x-32)
yes
00001104-0000-1000-8000-00805f9b34fb IrMC Sync
00001105-0000-1000-8000-00805f9b34fb OBEX Object Push
00001106-0000-1000-8000-00805f9b34fb OBEX File Transfer
0000110a-0000-1000-8000-00805f9b34fb Audio Source
0000110b-0000-1000-8000-00805f9b34fb Audio Sink
0000110c-0000-1000-8000-00805f9b34fb Remote Control Target
0000110e-0000-1000-8000-00805f9b34fb Remote Control
0000111e-0000-1000-8000-00805f9b34fb Handsfree
0000111f-0000-1000-8000-00805f9b34fb Handsfree Audio Gateway
0000112f-0000-1000-8000-00805f9b34fb Phonebook Access (PBAP) - PSE
00001132-0000-1000-8000-00805f9b34fb Message Access Server
00001133-0000-1000-8000-00805f9b34fb Message Notification Server
00001200-0000-1000-8000-00805f9b34fb PnP Information
00001800-0000-1000-8000-00805f9b34fb Generic Access
00001801-0000-1000-8000-00805f9b34fb Generic Attribute
0000180a-0000-1000-8000-00805f9b34fb Device Information
0000180d-0000-1000-8000-00805f9b34fb Heart Rate
00001826-0000-1000-8000-00805f9b34fb Unknown
00005005-0000-1000-8000-0002ee000001 Proprietary
347b0001-7635-408b-8918-8ff3949ce592 Proprietary
usb:v1D6Bp0246d0542
/org/bluez/hci0

[decodeais]

I made a lot of updates in the installation. Now the installation script works with Linux too.
I tested the software yesterday with Rouvy, it worked perfect.
I remember My first Bluetooth stick I bought for my old raspi2/3, it had a similar behavior. It told always something like headset or audio. I bought a hew one. That worked out of the box.
I think you should check with a bluetooth sniffer programm on the smartphone. Did you check a connection to a Tablet?

!!!!!!!!!!!!!!!! I have a idea. The reason will be that you started the program direct. you have to switch the bluetooth driver:

#-------------------------------------------------------------------------------
# If bluetooth required, stop service and enable for FortiusAnt
#-------------------------------------------------------------------------------
if [ $bluetooth == 1 ]; then
    echo Stop standard Bluetooth Service
    sudo service bluetooth stop
    echo Enable Bluetooth for FortiusAnt
    sudo hciconfig hci0 up
    if [ $? != 0 ] ; then
        Red='\033[0;31m'
        NC='\033[0m'
        printf "${Red} hciconfig failed, press enter to continue: ${NC}"; read x
    fi
fi

it part of FortiusAntMain.sh,

[WouterJD]

If you have any questions about git feel free to ask me.

Hi @zapiens good to hear.
Yes I have a question for you: as a non-experienced git-user I have put the executables and documentation in the repository.
After 4 years of development and uploading nex .exe and .pdf files, this pollutes the repository.

I would like to remove all old versions and then
Either continue as I did, so many new versions are not expected
OR proceed in a better manner where only the last .exe and .pdf are published.

Your recommendations are appreciated; if it's worthwhile we can start a new issue for that.

[decodeais]

I understand your problem. I think this Issue is really much to long now. Only the script to build the wxPython wheel and the links to the wheels should stay.
Have you an Idea how to treat our Wayland fork?
This changes had so many aspects, that it was a real problem.
I have great respect for your work, so I wanted to help you to make it possible to install FortiusAnt on the actual OS.
With the original installation script it is impossible now.
Sorry, this was the reason the script "exploded".

[zapiens]

Hi @WouterJD again thanks for all your work on this.

A principle I have used when working with git is to never check in generates. Generates should go to a storage instead.

Yes a repo gets bloated with time as binaries get checked in. To remove generates in the repo the history needs to be rewritten and thus all hashes after the changed point will be rewritten. In a small project with few developers I would communicate that and tell everyone to create pull requests for serious work done. After that freeze the repo and rewrite history and this will be a "new" repository that everyone needs to clone from (or fetch --force).

If a larger project I would create a new rewritten repo and tell everyone to use that one instead, the old could be around for a while for archaeological reasons ;)

[zapiens]

Thanks @decodeais for the suggestions. I brought up the bluetooth connection on both machines with bluetoothctl to have total control over what was going on. The connection was very unstable. It connected and then disconnected. Sometimes it stayed connected for longer time and I could send files back and forth.

When you say you use a bluetooth stick do you mean you omit using the built in bluetooth in RP and computer?
If this is the case why not buy ANT+ sticks instead?

[decodeais]

It was for an old Raspi without BT and a Tablet without built in Ant plus. Ant sticks are a better than BT but more expensive. My wife is happy rp2 and two ant+ sticks.