-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
InstantMeshes plugin development #2
Comments
You are welcome.
No, at the moment there are no nodes compatible with the InstantMeshes quad mesh output.
You are welcome to contribute. What do you have in mind? Here are the basics on how to implement new nodes:
Runs aliceVision_exportMVE2.exe with all parameters. {allParams} is only supported by alicevision plugins. For external programs cli parameters need to be adjusted (example):
Runs mogrify.exe with defined parameters
It is possible to implement new nodes in python without running an external program from the cli. Here is a good examples for pure python nodes: https://github.com/alicevision/meshroom/blob/develop/meshroom/nodes/aliceVision/SketchfabUpload.py References on different supported GUI elements for nodes can be found here
I am happy to share some knowledge (although I am still learning myself and mainly work on the documentation). There is no lack of ideas for new features, but you are always welcome to implement your own projects. Implementation notes: If you have any questions, feel free to ask. |
Wow thanks for the quick reply and all the information. I need some time to go through all of this. I assumed that the Texturing node can't handle quads and i think it's not trivial to implement this. You are absolutely right that implementing native stuff in c++ or python is the best way to go. So could you point me to some documentation on how to setup a development environment for Meshroom under Win7 with the VisualStudio Community Edition? My goals/dreams with meshroom:
Btw, i'm a hobby photographer since 10 years, so i would happily share my photo/image processing skills to. I procduced many 360 HDRi envirment maps by myself and also coded an own HDR processing tool. I could also imagine to contribute some of my own SFM image sets with a CC0 license. Phuu, enough for now. :) |
It might be possible to use https://github.com/caosdoar/Fornos to bake the high resolution texture from the Texturing node to the low poly from the InstantMeshes node. Having a tight InstantMeshes plugin (maybe even with preview) would be interesting. This is something we could discuss with the main MR devs.
Sure, I put this wiki together: https://github.com/natowi/meshroom_external_plugins/wiki
Support for normals is wip alicevision/AliceVision#578
I think there is some basic support for this
Sounds useful and not too difficult to implement (especially since you have experience in this area) There are some existing nodes related to HDR: LDRtoHDR and HDRI 360 stitching (will be available with the next release or when building from source)
That is interesting, too
I am working on a guide "capturing images for photogrammetry" for the Meshroom documentation (to reduce bug reports from new users that are based on unsuitable image datasets and provide a general guideline). If you are interested, I am happy to add you to the project.
A HDRI / exposure brackets sample dataset might be interesting. |
Sorry for comming back a little late, had not much time the last days. I added some info to your original top-level comment, but i don't know if i have done it in a good way. Want to hear your thoughts an that! For now, only the 'Triangle' mode works fine. The two Quad modes are working only sporadicaly. Seems there are problems with wrongly orientated and/or non-manifold faces. That would be the next thing i try to fix. Added/changed some commandline params and the descriptions. Maybe you can review that too A come back in a few hours to comment on your last post, i'm a little busy at the moment Thanks for all your helpfull information |
Good work. You are welcome to open a Pull Request for the InstantMeshes plugin to contribute your improvements. The comments can be re-written later on.
That is neat, nice find. I however did not yet get the IM node to work with the Texturing node.
There is no hurry, take your time. |
Thanks. Unfortunatly, i'm not familiar with Git, used mostly TFS and SVN in the past. Must learn howto do a propper Pull request first. Don't want to mess up your repo.
Ohh, that's bad. Did my IM node caused problems or did the Texturing node throw any error? (With "Remesh mode=Triangle" it worked every time i tested it. I used only tiny meshes <50k) |
Go to a file and click on the pen button to edit, this creates a new branch for you to edit. You can than save edits and later create a pull request. (#) No need to worry about messing up something. I used this dataset for testing https://github.com/alicevision/dataset_monstree with three images The quad mesh option takes forever with Smoothing iterations at 2. Set to 0 for fast computation. Works with Texturing. |
Cool, will try that after getting it a little bit more robust. Maybe tomorrow.
I assume you keept all settings at default? The DepthMap with downscale=2? Or have you used a different resulotion? Nevertheless, i think i found the main problem 😄
I will try to find a way to remove the self-intersections, maybe using some third-party lib. Hopefully, i come up tomorrow with a working Pull request 😉 |
Short update: But i stumbled accross MeshFix, a simple (award winning) commandline tool to repair the common defects in 3D scanned meshes. Unfortunatly, it needs the mesh to be a single closed object and that's very rarly the case using photgrammetry, cause you need a closed scan with the bottom of the object and totaly separeted from the background. 😞 (AFAIK, nearly impossible to get in Meshroom without manual cleanup in an external tool like Meshlab.) Luckily, i found PyMeshFix, a python wrapper for MeshFix, that make it possible to run the various repair steps separetly. So i want to give it a try using ONLY the repair self-intersections step. 🙏 If everything runs smooth, i would come up with a PoC in a few hours. |
Yes, I also found MeshFix, it produces good results but not on all meshes. It works best on mostly closed meshed. There is also Meshlabserver that can be used to run Meshlab from cli. It might be interesting to write some nodes that access Meshlab. https://stackoverflow.com/questions/20797535/meshlab-invoke-meshlab-command-line-version |
Hmm, my 'few hours' statement was a little bit too optimistic. Had problems removing the (in our case) unnesessary UI dependecies in PyMeshFix. But akaszynski was so kind to deliver a new package without the dependencies. 😄 @akaszynski Thanks again for that
Yes, thought about that too. Used Meshlab sometimes in the past and i think i have a little bit of understanding of the various cleanup and remeshing options in Meshlab. Could possibly be usefull, albeit the functions in meshlab seem a little bit outdated compared to the more sophisticated algorithms in other tools. Btw, found a nice research paper for an algorithm removing self-intersections (and as a side effect also non-manifolds) without the need to have a closed mesh. (As far as i'm understanding it) Ok, now i try to integrate the PyMeshFix functions and come up with a new version the next days |
Quick update: With the new PyMeshFix version 0.13.4, i managed to call the clean self-intersections separetly which, luckily, did excactly what i hoped for: cleaning the mesh without generating a closed one. 😄 The downside is, we now have two dependencies, PyMeshfix and numpy, that need to be copied into the meshroom 'lib' folder. Maybe i could make this optional. Now a i have to do some more tests with bigger/complexer meshes and cleanup the code a little bit (had to completly rewrite the OBJ load/save stuff). I will definitly try to do a Pull request with the new 0.4 version today. |
Ahh, one question: So my quesition is, did you know any cli tools that can bake normals from a highpoly to a lowpoly mesh and be able to use obj as input/output? |
Take a look at this list: https://gist.github.com/natowi/a8d6c1fffb91e58be8f10c45f21ba85f |
Hi, sorry for not replying so long. Bussy at work and a good friend visited me for a few days. Yes, i've taken a look at the fornos source code and there is sadly no cli handling implemented. Maybe i could do that, but for a quick solution i would give xNormal a shot. It has also no 'real' cli handling, but it's possible to use a xml file to control the parameters. So that's the next thing i like to try. Sooo, back to the InstantMeshes plugin. I've attached version 0.4 that now has detailed installation instructions included. I will do a pull request next. |
hello natowi & djoerg I got the new 0.4 version working with the texturing node using the 'outputMesh' output but not the 'outputInstantMeshes' output. Is there something I am missing? |
@barefoots outputMesh is the Meshroom compatible obj output, outputInstantMeshes the original output from InstantMeshes. |
aaaaaaaaaaaahhhh thank-you! |
@djoerg would you be interested in making a (WIP) PR in the main Meshroom Repository? Then we can clean up the code/comments and add all the dependencies to the requirements. |
@djoerg Thanks for your contribution,it looks like you have so many ideas about improving this plugin ( and not finished ).I also want to do something to improve it, could you assign me some of your tasks? and if possible some suggestion about how to implement them well. |
@natowi HI natowi, which python version are you uisng? I received error messages when runing " pip install pymeshfix ", it seems that there is no supported version of vtk for python 38. vtk from pypi:https://pypi.org/project/vtk/#files |
@TigerVersusT take a look at the source code, in the comments there are install instructions. (line 55+) |
oh, thanks! |
From natowi/meshroom_external_plugins#2 (comment) Some comment cleanup. InstantMeshes 0.4 code by https://github.com/djoerg
@TigerVersusT sorry, I forgot to answer you. Did the respond by ChemicalXandco help you? Yes, it would be good to write the debug messages into a log file. |
@natowi It's OK, and I will implement this feature as soon as possible |
[ ](url) |
@TigerVersusT that´s great! You are welcome push this to the instant meshes branch |
@natowi oh, I'll do this later, because I have some other features to implement for my bachelor’s degree |
@natowi Hi,natowi! Can I ask you some other questions? I really need to freeze my meshroom into executeable , and I used command "python setup.py install" to do this but find out that there is no dlls for Alice Vision framework. Could you point out where am I wrong? And how can I release my version to others( just like the official prebuilt binary )? Thanks. |
@TigerVersusT Meshroom is just the GUI for alicevision. Building Meshroom is tedious work, that´s why we are working on creating auto-builds using appveyor. You can download the latest win binaries QTOIIO, QMLAlembic, Meshroom, but for Alicevision the auto-build is not yet complete (some files missing) but you can build it yourself or try my build from a few months ago. For QtAliceVision we do not have an auto build yet. If you did not make any changes to Meshroom or alicevision, you do not need to build your own binaries. You can simply place your new nodes as py in the node folder and Meshroom will create pyc files for you. You can use the official binaries for that.
That is interesting, what are your plans? |
@natowi Thanks a lot for your information, may be I should reconsider my original plan of releasing my work to others. And the "Other features " currently is just 3D object compressing and decompressing( for test ) nodes using Draco, and for some reasons I also want to implement your original idea of "baking the high resolution texture from the Texturing node to the low poly from the InstantMeshes node" mentioned above。 |
No need to abandon the idea. I´d recommend you to fork meshroom and upload your changes to your fork. When you are done, we can see if we can include your work in the main project. Sadly I have not yet found a suitable texture baking tool. You can check out the wiki I linked to. There you can find helpful information and some interesting tools Btw. what is the topic of your thesis? |
Well , it.s "integration of 3D reconstruction software", and in my version of meshroom, except of instant-meshes node, I also changed some UI elements to call the python wrapped instant-meshes module and it is not suitable to be included |
@natowi Hi! natowi, I've met a wired problem: I followed your guide to complie the latest meshroom source code, the execuatable did worked but crashed when executing the "feature extraction" node( however it works sometimes ). The source works well for the same dataset. Is there anyting I've missed? Or have you ever met such problem? |
@TigerVersusT Yes, I also had some strange behaviour with feature extraction. I think there is already an issue for that. |
@natowi Thanks a lot!this is my source code |
You are welcome. I´ll take a look at your code later today. |
@TigerVersusT It worked (compiling) |
@natowi Oh, how about the complied version? Does the feature extraction node works fine? |
At the moment I only compiled Meshroom with your modifications |
👍I'll wait for your good news |
I need to recompile alicevision, as my last build is outdated |
Ok, everything did compile, but your nodes throw errors: ERROR:root:Error on node computation: Error on node "DracoEncoder_1": Do the nodes work in your dev environment (starting meshroom from the uncompiled python code)? Or did I misplace some of your files? However I did not experience any random crashes in FeatureExtraction, but it took a little longer than normal. Strange... I´d recommend you to better separate gui/python code and c++ code, like it is being done with alicevision and meshroom. This simplifies the build process and results in a better code structure. |
sorry for not repond immediately, it was too late for me yesterday. Ok, everything did compile, but your nodes throw errors:__ That's because it's a CLI based nodes, you can test or use it if you like Btw, how is the Retopology button supposed to work? hope you don't laugh at me, this feature was designed before I know anything about meshroom pipline( so it disobey the priceples of SE ), it works by cliking the 'retopology' button after the defualt pipline is finished, then a specially named obj file will be found in the {cache}{texturing}{unique id} .In case you have interest in the retopology module, I just exploit source codes of intantMeshes, delete UIs and refactor them to fit my goal. There is a VS .sln file in the source folder you can test it if you like. However I did not experience any random crashes in FeatureExtraction, but it took a little longer than normal. Strange... Yes, I think my code works fine, but the compiled version always crashed at the second node, may be there is some enviromental problem because even the latest source code crashed after compilation. I´d recommend you to better separate gui/python code and c++ code, like it is being done with alicevision and meshroom. This simplifies the build process and results in a better code structure. Thanks, something in my code does disobey principles of SE ( especially the retopology module ), I will refactor it later. |
What are the versions of your environment modules? I have: python 3.6.2 (64bit) and the exact versions specified in the requirements. |
The only difference is my python version is 3.68 maybe I should change to that to compile again, btw, can you send me your compiled version? This is my email address: 2647877536@qq.com or you can try 13550072794@163.com |
I tried with python 3.62 it crashed again, maybe I should recompile the AliceVision |
Ok, here are two builds you can try: exe.win-amd64-3.6-allmods.zip (Includes all your modifications + numpy+pymeshfix) exe.win-amd64-3.6-nodesonly-numpy-pymeshfix.zip (python nodes only + numpy + pymeshfix) (The easiest solution would be to take the official 2019.2 build, add the numpy and pymeshfix folders to the lib folder, copy the python nodes to the meshroom node folder and copy the exe files into alicevsion. No building required) |
cool! I'll try it now and reply you later. |
@natowi ,I test with the 'allmodes' one, and I find out that alicevision in the official build is not compatible with your compilation. After changing that to my self-compiled alicevision, everthing works fine( tested for three times). Thanks a lot for your help these days, and I think I'm ready for the next feature( hope you haven't finished them all )😄. |
Sorry, I already assumed you used your own compiled alicevision version. |
Hi,
first, thanks for trying to bring more features to MeshRoom. Highly appreciate it!
I'm curious if the obj format that InstantMeshes exports have ever worked on the Texturing node.
If've tried using InstantMeshes manualy, before i found your plugin, and the Texturing node always failed to load it. Sadly it's the same with your plugin.
After fiddling around a little bit, it seems to me that the Texturing node can't handle quad meshes. After converting the IM output to a pure triangle mesh via Meshlab and exporting it without normals or color, the Texturing node works. Including normals also leads to failure!
I've looked at the exported obj in an text editor and it turns out that IM is also exporting the mesh with normals!
How have you used your InstantMeshes node inside Meshroom?
Have you ever feed it directly into the Texturing node or have you put some other nodes in between?
Or, maybe it worked in an older version of MeshRoom? (I tried it with the 2019.2 version)
I really hope there's a solution for this. InstantMeshes works so well for automatic retopology, i would love to use it in a MeshRoom workflow.
Btw, i'm a software developer and i love the fact that it's so easy to implemet new nodes with python, so i would like to implement some own nodes or wrap some external CLI tools into nodes, but i have nearly no knowlegde about MeshRoom. It seems to me, that you have a fairly good understanding how MeshRoom works. Maybe we could work together (or share some knowledge) bringing more features to MeshRoom?
The text was updated successfully, but these errors were encountered: