Skip to content
This repository has been archived by the owner on Feb 9, 2021. It is now read-only.

Move to OCC instead of FreeCAD #268

Closed
dcowden opened this issue May 17, 2018 · 97 comments
Closed

Move to OCC instead of FreeCAD #268

dcowden opened this issue May 17, 2018 · 97 comments
Labels
CQ2 enhancement Priority:high

Comments

@dcowden
Copy link
Owner

dcowden commented May 17, 2018

@jmwright @adam-urbanczyk @fragmuffin

@adam-urbanczyk has a branch that moves most CQ functionality to use OCC directly, rather than the FreeCAD layer.

I'd like to begin the process of moving that into CQ main version.

Opinions on how to manage the reverse compatibility are welcome. For now, we'll probably use the OCC that comes with FreeCAD already.

My thinking is that we make an initial move, and then slowly move features from FC to OCC, rather than a huge push that changes it all, but what do you guys think? Maybe we should just move it all since @adam-urbanczyk has most if not all tests passing?
@dcowden dcowden mentioned this issue May 17, 2018
Closed
@osterwood
Copy link

I've had good success using Adam's OCC branch. As the code base is merged in, you may want to be aware of this patch I made to support cqparts' GLTF exporter:
CapableRobot@eae28b8

@dcowden
Copy link
Owner Author

dcowden commented May 17, 2018

@osterwood thanks for the information!
@adam-urbanczyk @jmwright do you have any reservations about making adams branch a real boy? If the tests pass, i'd be ok doing a new release for it.

@fragmuffin @jmwright from a cqparts viewpoint, and cqfm viewpoint, i would think a two step process would be best-- first, a new OCC release that seems to work, and then after that changes to cqparts and cqfm to use the new one?

@adam-urbanczyk
Copy link
Contributor

@dcowden I am in favor.
@osterwood looks interesting, I'll try to merge it if you don't mind. Why is numpy actually needed?

AFAIK there is no PythonOCC bundled with FreeCAD (at least not on my current machine). FreeCAD has a handwritten wrapper for the OCC kernel. I think what we need to begin is an API for selecting the impl module.

@osterwood
Copy link

numpy is just for the matrix convenience methods (shape, transpose, flatten). The logic could easily be embedded within the file. The _print methods can be ignored -- they helped with debugging but are no longer called.

transposed_list() is a surrogate for cqpart's call to transposed().A in the GLTF exporter. I forked that as well and added a fallback call, see CapableRobot/cqparts@4c9f1c8

Definitely not the cleanest way to "decouple" cqparts from the underlying implementation within cadquery. I made the same sort of change here CapableRobot/cqparts@58d237f for the shape exporters -- having cqparts call the cadquery export methods, if the existing & direct call to freecad fails.

@dcowden
Copy link
Owner Author

dcowden commented May 17, 2018

@adam-urbanczyk oh, i didnt know that your OCC branch was based on a freecad-specific OCC wrapper.

How much flexibility do we gain moving from FreeCAD objects to FreeCAD-authored OCC wrapper objects, vs moving to pythonOCC?

I have a love-hate releationship with pythonOCC. I love the development speed, the breadth of the project, and the people ( Jelle and Thomas), but I've historically found developing with it difficult. But its typically not been pythonOCC's fault-- its been OCC itself: its a bear to use at the base level.

I know for sure I need the direct OCC objects to do the historical queries I want to do

@adam-urbanczyk
Copy link
Contributor

adam-urbanczyk commented May 17, 2018 via email

@dcowden
Copy link
Owner Author

dcowden commented May 17, 2018

@adam-urbanczyk oh ok i was confused, sorry!

How will we distribute pythonOCC along with a cq version that needs it?

@gntech
Copy link
Contributor

gntech commented May 17, 2018 •
edited
Loading

If we want to distribute cadquery through pypi and bundle pythonOCC and the OCC binaries with cadquery perhaps we can do something similiar to the opencv-python package on pypi. It bundles prebuilt binaries of opencv in the pypi-package.
See their repo here: https://github.com/skvark/opencv-python

If we do it this way we can install via pip install cadquery

@adam-urbanczyk
Copy link
Contributor

@dcowden @gntech for me the only workable option is conda; pip is not really up to the task of distributing non-python packages (i.e. oce ). I already have a working conda package for cq-occ ( https://anaconda.org/cadquery/cadquery-occ ).

@dcowden
Copy link
Owner Author

dcowden commented May 17, 2018

Alright then! With no objections then let's move forward with integration of Adams branch.

@zbynekwinkler
Copy link

Will this mean that cadquery will not be usable from within freecad anymore?

Would it be possible to use the upstream version https://anaconda.org/pythonocc/pythonocc-core or does it have some problems? I see it has got builds for all the common platforms.

@zbynekwinkler
Copy link

Now I've found one more https://anaconda.org/cad/pyocct - it seems Freecad is moving towards using this package (see https://github.com/FreeCAD/FreeCAD_Conda).

@dcowden
Copy link
Owner Author

dcowden commented May 18, 2018

@zwn we definitely have to make sure cq can still run in freecad for some period of time, for cadquery freecad module. But I think the long term plan is to make our own gui, perhaps based on electron.

The perfect scenario would be if freecad actually used pythonocc, because then we could run in either environment with the same code. But i am afraid that won't happen. If it doesn't, we would have to support two codebases... One for cq on freecad and one for cq on OCC. This is not something I want to do long term. The reason for the move away from freecad is that there are tons of limitations and bugs we can't work around.

@gntech
Copy link
Contributor

gntech commented May 18, 2018

I have installed the cadquery-occ package from https://anaconda.org/cadquery/cadquery-occ for some testing and found that the installation instructions was lacking a bit.

On that page it says install with conda install -c cadquery cadquery-occ

However, since there are some dependencies from other channels needed this command was not enough. The following command activates the necessary channels to get the dependencies and install cadquery-occ.

conda install -c conda-forge -c DLR-SC -c oce -c pythonocc -c cadquery cadquery-occ

@adam-urbanczyk If you agree, perhaps this could be added to the cadquery-occ installation instruction page?

@adam-urbanczyk
Copy link
Contributor

@gntech Thanks for looking into this. I think that the line on anaconda.org is auto-generated, not sure how to change it.

Proper instruction can be found here: https://github.com/adam-urbanczyk/cadquery

@fragmuffin
Copy link
Contributor

😴 💤 ‼️ 😮 I'm awake!
Yep, I'm in!

I've created a "OpenCascade (OCC)" milestone for cqparts.
All changes will go in to the feature/freecad-occ branch until it's ready to be merged & released.

@osterwood thanks for your good work.
If it's alright with you, I'll cherry-pick what you've done to give me a head start!

@dcowden
Copy link
Owner Author

dcowden commented May 21, 2018

@here what do you think about this roadmap:

  1. Create occ branch in dcowden/cq, starting with Adams version

  2. Validate that tests pass

  3. Set up ci loops on the branch to build conda packages on release. ( I like the way we do that on the main branch... A release is cut whether a tag is present)

  4. Announce availability for those interested.

  5. Make a docker image for those who want to run that way (I'll take this one)

  6. Start work on a cq gui. @jmwright, what are your thoughts about the best platform upon which to do that?

  7. Run the branch in parallel with master until the library is solid and the new GUI works well

  8. Merge non freecad cq in to master. Hopefully at this point freecad is using OCC , and it's possible to run cq inside of freecad, by adding pythonOCC only.

History teaches that cq was not of much interest until there was a gui. It really took off once Jeremy made cadquery freecad module. Consequently, I think we have to have a functional GUI before we can try dumping Freecad.

The shortest path to a working GUI is probably to use the starting point from the pythonOCC samples, which in turn means using wx if I recall.

You can see A few at https://github.com/tpaviot/pythonocc-demos?

I can imagine building a gui starting from these pretty quickly.

@zbynekwinkler
Copy link

I see that cadquery-occ package is based on pythonocc-core (see https://github.com/adam-urbanczyk/cadquery/blob/cq1_pythonocc/conda/meta.yaml).

Freecad is using (py)occt (see https://forum.freecadweb.org/viewtopic.php?t=12920) - according to the posts even 0.18 release of oce has some bugs already fixed in occt 7.2.

Is cadquery aiming at using pyocct or pythonocc-core?

@zbynekwinkler
Copy link

I have been able to install https://github.com/adam-urbanczyk/cadquery on Windows 10:

  1. clone the repository
  2. run conda build . -c pythonocc -c oce -c conda-forge -c dlr-sc in conda subdirectory
  3. install cadquery-occ by running conda install cadquery-occ --use-local -c pythonocc -c oce -c conda-forge -c dlr-sc

There is a gotcha where the built package seems to have version equal to 0.

@dcowden
Copy link
Owner Author

dcowden commented May 21, 2018

@zwn good question. I didn't even know about pyocct until you brought it up. In conversation, i've been talking about pythonOCC, but that's only because its the one i'm familiar with. It's been around forever.

As mentioned in the article you referenced, the most notable difference is that pythonOCC is based on OCE, not OCCT. I'm not sure I care too much about that distinction.

if freecad is moving to pyOCCT, that would be good news for freecad/cq interoperability, but bad news for @adam-urbanczyk 's branch, which is based on pythonocc.

i'd be interested in thoughts on the apis themselves. pythonOCC started life as a simple set of auto-generated bindings, much like ( it appears ) pyOCCT is. However, over time, the pythonOCC guys have spent a lot of time tweaking the bindings to solve memory problems, garbage collection and such. I wonder if pyOCCT is doomed to repeat that work?

@zbynekwinkler
Copy link

http://pyocct.readthedocs.io/en/latest/extra.html#pythonocc-comparison but I have no idea why there are two different bindings :-(.

In pythonOCC there is also a qt based gui, see https://github.com/tpaviot/pythonocc-core/blob/master/src/Display/SimpleGui.py. I have not found anything similar in pyOCCT.

I'd just go with pythonOCC for now since it seems easier (a lot has been done already).

@dcowden
Copy link
Owner Author

dcowden commented May 21, 2018 •
edited
Loading

@zwn reading the front page of pyOCCT, this comment:

As new versions of OpenCASCADE are released, it may be better to patch up the existing bindings rather than rerun the binding generation tool. The automated process still requires a handful of manual patches to capture edge cases

Causes me to think they have a long path in front of them, and that it is very likely the same path pythonOCC has already walked. I think pythonOCC has a LOT of tweaks to their bindings, developed over literally a decade now. My hunch is that pythonOCC is the stronger play.

I do love how pyOCCT eliminates the use of handles, but i think the libraries' stability is more important. Plus, since @adam-urbanczyk already incorporates pythonOCC, we are a lot closer to a solution.

@adam-urbanczyk
Copy link
Contributor

@zwn @dcowden

I think at this point we need some clarifications of the terminology : ).

First of all there is OCCT ( Open CasCade Technology ) an open source C++ CAD kernel which is used by e.g. FreeCAD. Current version is 7.2.

There is also OCE (https://github.com/tpaviot/oce) which is a fork of OCCT (I guess to fix some bugs). Current version 0.18.3 is based on OCCT 6.9.1

Then there are python bindings:

  • PythonOCC is based on OCE
  • FreeCAD wraps parts of OCCT by hand and exposes as a python module
  • PyOCCT is a quite new project that exposes OCCT 7.2 using state of the art pybind11 package (as opposed to SWIG and PythonOCC). To my understanding there is no relation of PyOCCT and FreeCAD.

I like using OCCT 7.2 as a basis, but I cannot find the binding generator in the PyOCCT codebase (maybe it is not shared on purpose; will open an issue).

Regarding api compatibility: my gut feeling is that it won't be a lot of work to migrate, both PythonOCC and PyOCCT are more or less 1:1 translation of the C++ API of OCCT/OCE.

Let me know if I missed something.

@dcowden
Copy link
Owner Author

dcowden commented May 21, 2018 •
edited
Loading

I like using OCCT 7.2 as a basis, but I cannot find the binding generator in the PyOCCT codebase (maybe it is not shared on purpose; will open an issue).

I think you can't find it because it doesn't exist. They say right on the front page that what's there is the result of manual work:

The automated process still requires a handful of manual patches to capture edge cases

This causes me to worry about using it as a basis. In the long term, i like how they handle 'Handle'-- the code is much more intuitive. I also think we actually want the bugfixes in OCE vs OCCT. The difference is not what it used to be, but i still think there's value there.

@adam-urbanczyk adam-urbanczyk mentioned this issue May 21, 2018
Closed
@adam-urbanczyk
Copy link
Contributor

@dcowden the way I read it is:
there is an automated script, but the resulting code requires some manual edits.

Anyhow, let's see what the author will reply.

@dcowden
Copy link
Owner Author

dcowden commented May 21, 2018

sounds good. regardless, i think the level of maturity has to be better with pythonOCC right?

@adam-urbanczyk
Copy link
Contributor

@dcowden yes, for now PythonOCC it is. I will keep an eye on pyOCCT

@zbynekwinkler
Copy link

PyOCCT is a quite new project that exposes OCCT 7.2 using state of the art pybind11 package (as opposed to SWIG and PythonOCC). To my understanding there is no relation of PyOCCT and FreeCAD.

I figured we can just ask: https://forum.freecadweb.org/viewtopic.php?f=10&t=28863

@jmwright
Copy link
Collaborator

I don't think we want to transfer the CQ 1.0 repo. We basically want to rebase and start fresh on Adam's OCC branch/fork, if I understand the consensus correctly.

@jmwright
Copy link
Collaborator

@justbuchanan I don't mean to shut the conversation down. I was just stating what I think I understand the group wants to do. I'm certainly open to discussion on transferring versus rebasing.

@fragmuffin
Copy link
Contributor

As long as the history is kept in the new repository (eg: git blame).

Also migrating cqparts to OCC
It would make sense to migrate cqparts to OCC in parallel with cadquery, and cqparts is also going to be moved to its own organisation https://github.com/cqparts (which @zignig very kindly set up)

I'll add the cqparts repo to the organisation tonight.
I think I'll clone the current repo (from https://github.com/fragmuffin/cqparts) and push it to the organisation, then merge the feature/freecad-occ branch to master.

Wiki & Issues
This will not transfer wiki pages, issues, or projects to the new repository.
I'm going to select key "occ and beyond" issues and attempt to transfer them using github-issues-import... I'll let you know how that goes.
I believe github wiki pages are stored in their own git repository, so could be transferred with a bit of a hack.
Does cadquery have a wiki? [... looks ...] oh!, it's only 4 pages... just edit, select-all, copy/paste 4 times, done ;)

@zignig
Copy link

zignig commented Oct 28, 2018

@adam-urbanczyk can I get an invite to the cadquery org please?

@fragmuffin
Copy link
Contributor

fragmuffin commented Oct 29, 2018 •
edited
Loading

I think I'll clone the current repo (from https://github.com/fragmuffin/cqparts) and push it to the organisation, then merge the feature/freecad-occ branch to master.

Cloning to Organisation
OK, I tried manually forking the cqparts repository, by creating a blank repository in the cqparts organisation, then pushing it from my checkout with:

git push --mirror git@github.com:cqparts/cqparts.git

(while it's there: see it here renamed to cqparts_deleteme)

However this means the loss of:
issues, labels, projects, comments, PRs, release notes, wiki pages

Transferring
I also tried transferring a dummy repository from my user account to the organisation.
Everything above was successfully transferred.

For cqparts, I think it's worth transferring 👍

@fragmuffin
Copy link
Contributor

@jmwright @dcowden
Another thing worth knowing, github seems to have a very effective redirect on transferred repositories.
Any url to the old repo's home https://github.com/fragmuffin/cqparts will redirect to https://github.com/cqparts/cqparts, which includes:

  • github urls to the old repository (issues, files, commits, branches, wiki pages, projects, releases, etc)
  • github pages,
  • even git commands (push, pull, fetch, and so on)

are all redirected to the repository's new location....
I was going to create a cqparts under my username to advertise "cqparts has moved", but that would brake all redirects.
If the re-direct has a expiry, then I'll consider making one then.

To move cloned repository urls, don't re-clone it, just run:

# check remote urls
$ git remote -v
origin  git@github.com:fragmuffin/cqparts.git (fetch)
origin  git@github.com:fragmuffin/cqparts.git (push)

# set new url
$ git remote set-url origin git@github.com:cqparts/cqparts.git

# confirm change
$ git remote -v
origin  git@github.com:cqparts/cqparts.git (fetch)
origin  git@github.com:cqparts/cqparts.git (push)

@dcowden
Copy link
Owner Author

dcowden commented Oct 29, 2018 •
edited
Loading

Good to know, Thanks!

I'm not opposed to transferring cq as opposed to starting new, but it seems like starting new is the right thing to do with cq because it is basically a new product, and we need to keep the old one running as is for a while. We will have two branches running no matter what

@jmwright
Copy link
Collaborator

I agree with @dcowden

We can copy over summaries and links to relevant issues from the CQ 1.0 repo. There are quite a few issues mixed in that are related to FreeCAD and not OCC, so I don't think we want to transfer all issues. We also don't want to redirect people to CQ 2.0 until it's at least as capable (and preferably as easy to install) as CQ 1.0. I think a manual redirect and sunsetting process of 1.0, while more work, is more appropriate here.

@fragmuffin
Copy link
Contributor

Sounds good!
I'm looking forward to things being a bit more clear instead of being scattered across all the contributors' forks.

Once the new repo' is ready, I'd like to port the gltf geometry exporter implemented in cqparts to cadquery (cqparts/cqparts#49).
But I don't see the point in doing so for the freecad build.

May the forks be with you 🖖 ... (I couldn't help it)

@adam-urbanczyk
Copy link
Contributor

OK I imported my fork into https://github.com/CadQuery/cadquery and deleted all branches but one. Does this look right to everyone? (If not we can always redo using different approach)

@dcowden
Copy link
Owner Author

dcowden commented Oct 30, 2018

Thanks Adam!
Is it possible to rename this branch master?

I think next we need to set up Travis and all the test automation

@adam-urbanczyk
Copy link
Contributor

Renamed to master

@jmwright
Copy link
Collaborator

jmwright commented Dec 3, 2018

@dcowden @adam-urbanczyk @fragmuffin @justbuchanan @RustyVermeer @zignig

We've got a lot of issues here on the CQ 1.0 repo that are tagged with CQ2, and those really belong on CadQuery/cadquery now. GitHub recently added a beta feature to transfer issues between repos, and from what I've read it's pretty safe. I suggest that we transfer the CQ2-tagged issues over to the CQ 2.0 repo, and from there they can be dealt with. I think the PythonOCC implementation has been done in a different way that @dcowden envisioned when he created those issues, so some issues may need to be closed with the wontfix tag.

Thoughts?

@dcowden
Copy link
Owner Author

dcowden commented Dec 3, 2018 via email

@jmwright
Copy link
Collaborator

jmwright commented Dec 3, 2018

@dcowden Yes, you will need to do the migration one issue at a time, unless you want to elevate my privileges on this repo. Then I can do it. I don't see the option for me on this repo, but I do on CadQuery/cadquery.

https://help.github.com/articles/transferring-an-issue-to-another-repository/

@dcowden
Copy link
Owner Author

dcowden commented Dec 3, 2018 via email

@dcowden
Copy link
Owner Author

dcowden commented Dec 5, 2018

@adam-urbanczyk can you please double-check that im an admin on Cadquery/cadquery?
I need admin to be able to transfer issues over from dcowden/cadquery. I can't seem to do that, so i may not be an admin.

@dcowden
Copy link
Owner Author

dcowden commented Dec 5, 2018

@here i added words to the readme on dcowden/cadquery to publicise the move to 2.0 at Cadquery/cadquery.

@adam-urbanczyk
Copy link
Contributor

@dcowden this is what I see. Are you sure that it is permissions related?
obraz

@Peque
Copy link
Contributor

Peque commented Dec 5, 2018

Maybe it is not possible to transfer issues from one organization/user into a different organization (?).

@dcowden
Copy link
Owner Author

dcowden commented Dec 5, 2018 via email

@dcowden
Copy link
Owner Author

dcowden commented Dec 6, 2018

@here looks like the best way to do this is probably to use this:

https://github.com/IQAndreas/github-issues-import

I'd probably close ones i think dont need to be there, and then import the rest. would that work?

@dcowden
Copy link
Owner Author

dcowden commented Dec 6, 2018

@adam-urbanczyk @jmwright I used the above tool to tranfer a few issues., for example, this one Its less than ideal, they all come over as comments by the person running the thing.

I'm also having problems using it for bulk-- its erroring out with some kind of authentication issue. But if we're ok with this format i can bring them over one at a time i think

@jmwright
Copy link
Collaborator

jmwright commented Dec 6, 2018

It seems fine to me.

@fragmuffin
Copy link
Contributor

The issues aren't displayed as elegantly as the originals, but the information is all there. 👍

@dcowden
Copy link
Owner Author

dcowden commented Dec 6, 2018

ok i'll move the others over.

@dcowden
Copy link
Owner Author

dcowden commented Dec 6, 2018

@adam-urbanczyk @jmwright @Peque ok i moved over all the issues I think are appropriate ( and a couple that might be on the fence. Be aware, the utility above works pretty well, but github has rate limiting that will produce random errors. There are several copies of issues that are closed, because when it was failing mid import i'd have to close and make a new one.

I'm going to close CQ 2.0 issues in this repo

@dcowden
Copy link
Owner Author

dcowden commented Dec 6, 2018

This has been closed in favor of the CQ 2.0 implemention at https://github.com/cadquery/cadquery

@dcowden dcowden closed this as completed Dec 6, 2018
@jmwright jmwright mentioned this issue Dec 19, 2018
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
CQ2 enhancement Priority:high
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests