Replies: 31 comments 30 replies
-
|
Hi everyone, Great idea to introduce ourselves here. I first want to thank everyone who contributed to GliderTools so far and set the ground! I am Soeren Thomsen, Physical Oceanographer. I have been working with Slocum gliders since around 2011, including the development of turbulence processing software for MicroRider measurements during my Bachelor at GEOMAR. For my PhD/PostDoc at GEOMAR I designed and carried out two glider fleet experiments off Peru (https://agupubs.onlinelibrary.wiley.com/doi/full/10.1002/2015JC010878, https://agupubs.onlinelibrary.wiley.com/doi/full/10.1002/2016GL070548) and Mauritania (https://bg.copernicus.org/articles/16/979/2019/). Big fan of open science. So all related datasets are quality controlled open access available on Pangaea (https://doi.pangaea.de/10.1594/PANGAEA.860727, https://doi.pangaea.de/10.1594/PANGAEA.860480) thanks to G. Krahmann and the data management team at GEOMAR 🙏. Please get in contact if you are interested in glider datasets in upwelling systems (i.e. off Peru, Mauritania, Namibia or Benguela). I can navigate you through the different available deployments available on Pangaea. Right now I work 50 % as OceanGliders.org Best Practice coordinator. I am willing to contribute to GliderTools i.e.to ensure that OceanGliders.org Best Practices i.e. on Delayed Mode Quality Control (DMQC) are implemented into GliderTools. I switched to Python in 2018 mainly to analyse ROMS/CROCO model output using xarray, dask etc! So far I have little experience in collective coding but motivated to learn on the fly! Looking forward to working with all of you! Best |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, I'm Callum Rollo, early career physical oceanographer and Python/linux tinkerer. I have a few years experience with Seagliders and want to drive more community development of open source tooling. I take the Argo community as a great inspiration, particularly packages like argopy. I haven't contributed to GliderTools yet, but aim to get it on conda-forge to increase visibility and ease of use. |
Beta Was this translation helpful? Give feedback.
-
|
Hello! Isabelle Giddy here. I am a Phd student physical/ bio-physical oceanographer. Love the philosophy and community in open-source code development. I use GliderTools frequently for my research and have been/ still am tentatively contributing :) - initially worked on the gridding module. Now working to fix errors and update some of the existing modules. My current focus is one of best practice in that I want to develop the code I use in my research into generic modules that facilitates asking future science questions with glider data and learn along the way. |
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone! I'm Marcel du Plessis, a postdoc at the University of Gothenburg. I worked with Seaglider data for my Masters and PhD and assisted in the conceptualisation and testing of GliderTools. I am new to contributing to the open-source community but I'm eager to learn and help it grow where I can! I've been using Python for a few years now and am extremely grateful to the community for making such amazing tools which I use almost every day. |
Beta Was this translation helpful? Give feedback.
-
|
Howdy, folks. I am Julius, an Assistant Research Scientist at Lamont/Columbia University. I have worked with many different obs platforms (but not gliders yet) in my career but am at the moment more focussed on working with Earth System Models. I really don't like to make the distinction between "Observational Oceanographers" and "Modellers", and assertions that one of them is just inherently more technically inclined. I firmly believe that future science needs both and that increasingly complex and interdisciplinary topics require our work to be reproducible, open, and collaborative. First and foremost I hope this effort will foster a community effort that avoids duplication of work and encourage more usage of the amazing data gliders provide to everyone from the 'veterans' to newcomers and even folks like myself, who don't work with this data on a day-to-day basis. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for your contribution Juls! 🙏 |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
Hi glidertools I'm Tom, marine observations scientist at the Cefas and the University of East Anglia (UK). My science background is in biogeochemistry and I have a particular interest in dissolved oxygen and optodes. I'm very happy to contribute to this effort. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @tomhull ! Nice to e-meet you! Great to hear that you are also working on oxygen and optodes! |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, |
Beta Was this translation helpful? Give feedback.
-
|
Hi @AnthonyBosse , you might enjoy pyRings (https://github.com/castelao/pyrings) a package to estimate the eddy-center. It considers the deformation of the eddy due to the translation during the survey. That can be an important factor since the gliders are relatively slow. It's old but should work fine. |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, |
Beta Was this translation helpful? Give feedback.
-
|
Hi Everyone, I am Dhruv Balwada, a physical oceanographer. I am currently a postdoctoral researcher at the University of Washington (not with the glider people here). My research generally focuses on ocean turbulence at the meso and submesoscales, and over the years I have really enjoyed describing "the shape of water" (essentially quantifying variability at different scales) among other things. I have only started working with glider data since late 2019, am almost completely new to this enterprise, and wanting and willing to learn. I transitioned to gliders after having spent some time working closely with the teams that were developing Pangeo, and so as soon as I started I looked for an open source toolbox. I was very glad to have found Glidertools, and am now even more excited by the rapid rise in engagement. I think these open source tools not only provide easy access to code and save huge amount of time, but also help build better frameworks to think in - essentially collecting together the best ideas from many people that rise out of collective trial and error. It also helps democratize the process, as I don't need to be working with the one group that "does things best" anymore, and experience can easily be shared and transferred. In terms of working with gliderdata, I am very interested in the mapping and visualization side of things. I feel that since gliders are starting to look at scales that have traditionally not been accessible to oceanographers, we need to reevaluate how we think and work with these spatio-temporally complex sampling patterns. I am also starting to wonder about what are the best ways to compare glider data with high resolution simulations, specifically thinking beyond the fact that the ocean models at high resolutions do well because the wiggles from the high resolution models are qualitatively starting to look similar to those from gliders. Looking forward to collaborating with folks here. |
Beta Was this translation helpful? Give feedback.
-
|
Hi! My name is Sarah Nicholson, i'm a Researcher at Southern Ocean Carbon-Climate Observatory (SOCCO) based at the CSIR, South Africa, Cape Town. Iv been working with Seagliders, Slocums and Wave Gliders since 2017 and I currently co-Lead glider activities at SOCCO. Im particularly interested in microstructure from gliders, twinned glider experiments and understanding the coupled physics-carbon system. I contributed towards conceptual development of GliderTools and am an avid user. Happy to continue to contribute where I can. |
Beta Was this translation helpful? Give feedback.
-
|
Hi! |
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone, My name is Tommy Ryan-Keogh, I am a postdoctoral research fellow at SOCCO-CSIR based in Cape Town, South Africa. I am an original contributor and code developer of GliderTools. I have developed other packages in python as a part of SCOR activities, and want to see GliderTools grow into a community tool that is constantly growing as our needs and requirements do. I have been working with Gliders since 2015 with a specific focus on bio-optics to examine biogeochemistry (phytoplankton production, carbon export). |
Beta Was this translation helpful? Give feedback.
-
|
Hello everyone, my name is Alice Pietri, I'm a physical oceanographer working at IMARPE in Peru. I've been working with gliders since I started my phD a decade ago in France (LOCEAN) and then in Germany (GEOMAR). I mainly have experience with slocums, from preparation to deployment to data processing. So far I've only been using Matlab but I have been meaning to switch to Python for some time now and I hope I can get involved and contribute. I'm very motivated to build a more open environment to facilitate glider data processing. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @AlicePietri , I was on that glider workshop at IMARPE a few years ago organized by Dimitri. The Carpentries have a nice introduction to Python (https://software-carpentry.org/lessons/) |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, I'm Luke Gregor. I am based at ETH Zürich doing a postdoc with a focus on global carbon fluxes and ocean acidification. |
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone, this is a great initiative! I am Svenja and a postdoc at Woods Hole Oceanographic Institution. I have not worked with gliders before but will likely start using/processing glider data sometime this year. After using Matlab for almost 10 years, I switched to Python when I started my postdoc in 2019 where I am working with large model outputs - have not regretted it!! The glider person at WHOI uses Matlab, so I am definitely keen to see where GliderTools is going and compare tools. Hopefully of course I hope to be able to contribute along the way. I got introduced to collective coding during OceanHackWeek, which was a lot of fun. Currently I am studying marine heatwaves, their depth structure and regional drivers and recently shifted my focus on the Northeast US continental shelf, which is the region where I will be analysing glider data. Looking forward to the interactions and discussion here and hope I can be of help! |
Beta Was this translation helpful? Give feedback.
-
|
Hi Svenja. marine heatwaves. and in particular their depth is a great current topic (in Australia anyway) and gliders can contribute in a fantastic way. many SST heatwaves are really, thin mixed layer events. The water column temp is, depth average, unchanged. Others are advection events or .... ocean glider data (and argo floats & modelling) can reveal the characteristics of a marine heatwave. we have a marine heatwave event based glider program and we see advection events; like the strengthening of the east Australian current bringing warmer water to the Tasmanian coast https://anfog.ecm.uwa.edu.au/slocum_portal/unit287/TasEastCoast-09Mar21/unit287_TasEastCoast-09Mar21.kmz |
Beta Was this translation helpful? Give feedback.
-
|
Hi Svenja (@sryan288), Great to have you in! Which glider platforms are used at WHOI? You research topic sounds exciting. Please check out https://ebuswebinars.wixsite.com/ebuswebinars where we have talks about upwelling systems every months. There are also old webinars to watch. Best Soeren |
Beta Was this translation helpful? Give feedback.
-
|
Hi Soeren (@soerenthomsen), thanks - will check out the seminars! As far as I know it is mainly Spray gliders that are used, which were developed at WHOI and Scripps (http://gliders.whoi.edu/instrumentation/spray/). Not having worked with gliders, I am not sure how the differ from e.g. Slocum - should probably read up on that once I start working with the data ;-) |
Beta Was this translation helpful? Give feedback.
-
|
Hi @sryan288 , |
Beta Was this translation helpful? Give feedback.
-
|
Hi GliderToolsCommunity, I am Lewis and I work as a robotics support scientist at the Scottish Marine Robotics Facility based at the Scottish Association for Marine Science. We work almost exclusively with Seagliders deployed around Scotland, the North Atlantic, and Arctic Ocean. Our data processing is mostly done 'in house' by Postdocs on various projects but we are now looking to adopt a toolbox for all processing and data delivery. GliderTools could be perfect for us. No-one in our team uses Python, but now may a good time to start. I hope I will be able to contribute once I have learned how to use the toolbox. For now, is there an IDE folks can recommend for beginners using this GliderTools? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! Going to try with Pycharm, and see how I get on, |
Beta Was this translation helpful? Give feedback.
-
|
Not quite as powerful (yet) as most IDEs but I can recommend checking out JupyterLab. This is used very widely on cloud computing instances like the pangeo one, but also on JupyterHub Deployments on various HPC centers, and you can run it locally. It makes it pretty neat to switch between systems, all you need is your webbrowser! |
Beta Was this translation helpful? Give feedback.
-
|
+1 for JupyterLab. I used Matlab's IDE before, then used PyCharm a lot, but have now switched to JupyterLab entirely for this kind of analysis. Much more interactive, and somewhat more intuitive for ex-Matlabers. B |
Beta Was this translation helpful? Give feedback.
-
|
+1 for JupyterLab too. That is what I used when I transitioned over from MATLAB. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks all. I'm finding Pycharm a bit cluttered and confusing. Jupyter, on the other hand, is really great! |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, |
Beta Was this translation helpful? Give feedback.
-
|
Hi @nlbeaird! Great you are joining the effort. Have you worked on specific variables ? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @soerenthomsen, thanks! I've mainly worked with standard (unpumped) CTD data, mostly to look at overflows ( eg: https://doi.org/10.1175/JPO-D-13-029.1 , https://doi.org/10.1175/JPO-D-12-094.1, https://doi.org/10.1002/2015JC011501) . Part of that was early work trying to infer dissipation of TKE from glider vertical velocity - something I'm still interested in. Recently I've been working with a student on some simple processing for active acoustics on gliders to look at zooplankon distributions (some messing around here: https://github.com/nlbeaird/glider_acoustics) |
Beta Was this translation helpful? Give feedback.
-
|
Great work @nlbeaird ! I am happy we connected here. I will get into contact with you also related to some OceanGliders.org Best Practice efforts! |
Beta Was this translation helpful? Give feedback.
-
|
Nice to see the community growing! Welcome everyone :) |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, This is Gui Castelão from Scripps. I'm a physical oceanographer, and I do part of the Spray data pipeline, from satellite communications (https://github.com/castelao/iridiumSBD) to the GDAC & public data files (https://spraydata.ucsd.edu). I developed the Python infrastructure for Spray, also using xarray. If there is interest, I would be happy to merge or at least work to keep cross-compatibility. I also developed a package for automatic QC of oceanographic data, including gliders, that could be easily connected here (https://github.com/castelao/CoTeDe). I've been involved with Open Source for a while, so I'm excited to see this movement here. |
Beta Was this translation helpful? Give feedback.
-
|
Great to have you on board Gui (@castelao)! This is motivating! I am sure there is interest to merge where it makes sense and/or keep cross-compatibility. I am happy to facilitate this discussion and started an overview figure for a new package structure. This could be a good starting point to discuss together what could be merged and which tools could be made compatible. I will start a new discussion threat soonish on this. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I am Lucas Merckelbach, an oceanographer at Helmholtz Center Hereon, Geesthacht, Germany. (formally know as HZG). Here I work in a small group mostly focused on processes in stratified shelf seas, in particular turbulence, in relation to the effects of offshore wind farms. Started to work with Slocum gliders back in 2006, then at NOC in Southampton, UK, which was also the time I ditched matlab in favour of python. I am the author of dbdreader (https://github.com/smerckel/dbdreader/), which is a python module to read the binary files generated by Slocum gliders. As I understand it, GliderTools lacks a proper support for Slocum glider data, so I am happy to put in my expertise to improve the interface to Slocum data. I also wrote the python module gliderflight (https://github.com/smerckel/gliderflight), which can be used to compute glider speed through water, as well as vertical water velocities. The speed through water is an important input parameter in the processing of microstructure data obained from gliderborne microstructure sensors. It may be of interest to some of you. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @smerckel, Great to have you onboard! Indeed it would be very important for GliderTools to be able to import Slocum glider data. I tried to make a first proposal for a new package structure here: #107. I assume these activities would also need to aligne with the new OceanGliders format mentioned here #49 and #50. |
Beta Was this translation helpful? Give feedback.
-
|
Note pyglider is now using dbdreader (optionally). dbdreader is much more robust and fast than the tools we were using before to do the raw conversion. |
Beta Was this translation helpful? Give feedback.
-
|
Hi all, I’m Louis Clement, I am a physical oceanographer and I work at the NOC in Southampton, UK. |
Beta Was this translation helpful? Give feedback.
-
|
I know that dude! Great to have you here, Louis! |
Beta Was this translation helpful? Give feedback.
-
|
Thank you Julius, great to see you around as well. |
Beta Was this translation helpful? Give feedback.
-
|
Hello everyone, |
Beta Was this translation helpful? Give feedback.
-
|
Welcome @fabien-roquet ! Indeed there must be a lot of similarities in how to treat the data from seals and gliders. For sure seals have even longer lasting battery packages! ;) |
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone, My name is Charlotte Williams, I am a Research Scientist in the Marine Physics and Climate Group at the National Oceanography Centre, working primarily with our fleet of Gliders (since 2015). I have been involved in >40 glider deployments as part of the shelf sea biogeochemistry project (https://www.uk-ssb.org/), as well as being responsible for the slocum glider data for the AlterECO project (https://altereco.ac.uk/). I have a multidisciplinary background and am interested in understanding biophysical interactions. I am particularly interested in using gliders to investigate oxygen depletion and dynamics in the coastal and shelf sea, and how this might respond to a changing climate. I have experience in using Aanderaa oxygen optodes that have been deployed on slocum gliders and am interested in finding the best practice to get the best possible oxygen data from gliders. I am also a co-WP lead on the EU GROOM II (gliders for research, ocean observation and management) project (https://www.groom-h2020.eu/). |
Beta Was this translation helpful? Give feedback.
-
|
Hi everyone, |
Beta Was this translation helpful? Give feedback.
-
|
The tools I use mostly are python and jupyter notebook with pandas and sqlalchemy & sqlite db. I think Julia language would also be worth to notice. |
Beta Was this translation helpful? Give feedback.
-
|
i haven't tried glider tools (insert embarrassed emoji ) we have our own publicly available tool called glidescope which can be use to browse data and plot things up. generally i use this to roughly explore data then load it into python for more detailed analysis.
If the problem is opening the netcdfs in python. then there is a work around for this and that is to load them in via xarray.
here is some code i use to import three net cdfs into pandas data frames.the key bit is the last lines where i pass the netcdf data into xarray and then doing a to_dataframe from xarray instead of from netCDF directly. i don't know why this is more error tolerant but the direct import from netCDF doesn't work but via xarray does.
import pandas as pdimport xarray as xrfrom netCDF4 import Dataset
# now get glider data gliderdatafilenameDec="IMOS_ANFOG_BCEOPSTUV_20191113T012730Z_SL287_FV01_timeseries_END-20191205T005735Z.nc"gliderdatafilenameJan="IMOS_ANFOG_BCEOPSTUV_20200123T021637Z_SL287_FV01_timeseries_END-20200224T233110Z.nc"gliderdatafilenameFeb="IMOS_ANFOG_BCEOPSTUV_20200227T032645Z_SL287_FV01_timeseries_END-20200327T232039Z.nc"# open the dataset
columns_to_drop =['PLATFORM', 'DEPLOYMENT', 'SENSOR1', 'SENSOR2', 'SENSOR3', 'SENSOR4', 'TIME_quality_control', 'LATITUDE', 'LATITUDE_quality_control', 'LONGITUDE', 'LONGITUDE_quality_control', 'HEAD', 'HEAD_quality_control', 'UCUR', 'UCUR_quality_control', 'VCUR', 'VCUR_quality_control', 'UCUR_GPS', 'UCUR_GPS_quality_control', 'VCUR_GPS', 'VCUR_GPS_quality_control', 'PHASE', 'PHASE_quality_control', 'PROFILE', 'PROFILE_quality_control', 'PRES', 'PRES_quality_control', 'DEPTH_quality_control', 'TEMP_quality_control', 'CNDC', 'CNDC_quality_control', 'PSAL_quality_control', 'DOX2', 'DOX2_quality_control', 'DOX1', 'DOX1_quality_control', 'CPHL', 'CPHL_quality_control', 'CDOM', 'CDOM_quality_control', 'VBSC', 'VBSC_quality_control', 'BBP', 'BBP_quality_control', 'IRRAD443', 'IRRAD443_quality_control', 'IRRAD490', 'IRRAD490_quality_control', 'IRRAD555', 'IRRAD555_quality_control', 'IRRAD670', 'IRRAD670_quality_control'] # columns to leave out of the dataframe to stop gobbling memory
gliderdataDec= xr.open_dataset(gliderdatafilenameDec, drop_variables = columns_to_drop).to_dataframe() #load netcdf into pandas dataframe via xarray#gliderdata.append(xr.open_dataset(gliderdatafilenameJan, drop_variables = columns_to_drop).to_dataframe())gliderdataJan= xr.open_dataset(gliderdatafilenameJan, drop_variables = columns_to_drop).to_dataframe() #load netcdf into pandas dataframe via xarraygliderdataFeb= xr.open_dataset(gliderdatafilenameFeb, drop_variables = columns_to_drop).to_dataframe() #load netcdf into pandas dataframe via xarray
Hope this helpsDennis
On Wednesday, 10 November 2021, 01:15:58 pm AEST, Neil Malan ***@***.***> wrote:
@dennisstanley64, have you experimented at all with getting the IMOS glider netcdfs to play nice with GliderTools? I've been messing around with this (with limited success, some things work, others don't), and was wondering if anyone else had trodden this path before?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks Dennis, our current glider workflow is similar with xarray and pandas, we have about half of glider tools functionality working (everything in the vertical is fine, horizontal not so much). We're hoping to write a reader for the IMOS format, will share if we get it right! |
Beta Was this translation helpful? Give feedback.
-
|
Dear @dennisstanley64 and @nmalan thanks for your input. We try to keep our discussions as organised as possible. This is the discussion thread. I transferred your discussion to a new topic. Please adapt accordingly (i.e. title etc). And remove the thread here. Thanks a lot. |
Beta Was this translation helpful? Give feedback.
-
|
hmm by horizontally you mean XY (east north) because mostly our data is indexed temporally. Which i think is more accurate . given we KNOW what time a reading was made but we don't KNOW where it was made, exactly. in the future others may model our flight date better than us and obtain a better approximation of position and we shouldn't pre empt that with our current arbitrary approximation of position. Time is what we know to be correct and so how measurements should be indexed IMHO .
On Wednesday, 10 November 2021, 02:17:15 pm AEST, Neil Malan ***@***.***> wrote:
Thanks Dennis, our current glider workflow is similar with xarray and pandas, we have about half of glider tools functionality working (everything in the vertical is fine, horizontal not so much). We're hoping to write a reader for the IMOS format, will share if we get it right!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
|
Beta Was this translation helpful? Give feedback.
-
|
Hi, I'm Jody Klymak, Professor at University of Victoria, and principal PI for https://cproof.uvic.ca. Apparently I am also our lead developer, and in particular checkout https://github.com/c-proof/pyglider which purports to make CF-compliant NetCDFs from your glider data using a short python script and yaml metadata. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I am Martin Mohrmann, Baltic Marine Researcher for the Voice of the Ocean Foundation. Originally from northern Germany, I did my PhD at the University of Gothenburg in physical oceanography and continue to work in Gothenburg for VOTO now. I have hardly worked with Gliders before, but I do it now and GliderTools is a great way to get started with the analysis. I also just wrote by first contribution. ;) |
Beta Was this translation helpful? Give feedback.
-
|
Great that you have joined Martin and thanks for supporting GliderTools! |
Beta Was this translation helpful? Give feedback.
and others
-
👋 Welcome!
We’re using Discussions as a place to connect with other members of our community. We hope that you:
build together 💪.
To get started, comment below with an introduction of yourself and tell us about what you do with this community.
Beta Was this translation helpful? Give feedback.
All reactions