Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Intermittent failures to download maven dependencies #93

Open
barbeau opened this issue Jun 10, 2014 · 26 comments
Open

Intermittent failures to download maven dependencies #93

barbeau opened this issue Jun 10, 2014 · 26 comments
Labels
bug

Comments

@barbeau
Copy link
Member

barbeau commented Jun 10, 2014

Starting around May 24th there appeared to have been intermittent issues with downloading maven artifacts from the nexus.onebusaway.org maven repository (which contains all previous releases of OBA, as well as dependencies required to build the project).

According to @alanborning, @bdferris is working on updating the machines hosting the artifacts.

As of 6/10, it appears that nexus has been migrated to a new machine (when you hit nexus.onebusaway.org from a browser, it now shows a different page), and some artifacts are available to download.

However, it appears that we're missing at least two artifacts from the nexus repo, which results in a broken build on a machine that hasn't previously built OBA (i.e., a machine that hasn't previously cached these artifacts).

If you run mvn clean install -U on a new machine, you get this error:

[INFO] onebusaway-quickstart-assembly .................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 31.737 s
[INFO] Finished at: 2014-06-05T15:10:48-05:00
[INFO] Final Memory: 29M/623M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project onebusaway-geospatial: Could not resol
ve dependencies for project org.onebusaway:onebusaway-geospatial:jar:1.1.12-SNAP
SHOT: The following artifacts could not be resolved: com.vividsolutions:jts:jar:
1.10, edu.washington.cs.rse:javaproj:jar:1.0.4: Could not find artifact com.vivi
dsolutions:jts:jar:1.10 in public.onebusaway.org (http://nexus.onebusaway.org/co
ntent/groups/public/) -> [Help 1]

Therefore, it appears that com.vividsolutions:jts:jar:1.10 and edu.washington.cs.rse:javaproj:jar:1.0.4 are the two missing artifacts. JTS 1.10 isn't available from Maven Central for some reason, and Javaproj by UW doesn't seem to be publicly available for download elsewhere, so users without a cached Maven repo can't build OBA.

A manual workaround is described here (to manually download a zip file containing these artifacts, pulled from my local machine, and install them yourself):
https://groups.google.com/d/msg/onebusaway-developers/OvI7qWOinME/Ouz2YMUAc-wJ

Discussions on the OBA users and dev lists about this issue:
https://groups.google.com/d/msg/onebusaway-developers/OvI7qWOinME/07pZlbG4RCEJ
https://groups.google.com/d/msg/onebusaway-users/mAxn6kednRI/DD61EAqCZC0J
@barbeau barbeau added the bug label Jun 10, 2014
@bdferris
Copy link
Member

We were in the process of migrating to a new Nexus instance on a new UW server, but that work wasn't completed yet. Unfortunately, it looks like someone switched the DNS entry prematurely. I will see if someone can switch it back?

@barbeau
Copy link
Member Author

barbeau commented Jun 10, 2014

@bdferris Thanks for taking a look at this!

Another workaround - @sheldonabrown has set up a maven repo hosting some of the missing artifacts, so you can now grab missing artifacts from Sheldon's repo using the following maven command format (vs. the manual method mentioned in the above lists):

mvn org.apache.maven.plugins:maven-dependency-plugin:2.1:get -DrepoUrl=http://developer.onebusaway.org/archiva/repository/internal/ -Dartifact=com.vividsolutions:jts:1.10

This should pull down the artifact from Sheldon's repo if nexus isn't up, and install it in your local maven repository. If you keep getting missing artifact errors for new artifacts, repeat the above maven command for each of them, and you should eventually get to a complete build.

@barbeau
Copy link
Member Author

barbeau commented Jun 12, 2014

From S. Morris Rose:
https://groups.google.com/d/msg/onebusaway-developers/OvI7qWOinME/tEW-pEUIfVgJ

I think that our Maven nightmare is over- it evidently was due to a machine being restarted with configuration that gave it the same IP as the box serving nexus.cs. My mistake- made back in December 2013 but laying in wait until late May. Sorry for the outage- it seemed to come and go, which made it a head-scratcher.

It seems fixed to me (i.e., I was able to do a complete build from scratch, and the nexus.onebusaway.org has been correctly pointing to the Sonatype repo for the last day or so), so I'm going to close this issue.

@barbeau barbeau closed this as completed Jun 12, 2014
@barbeau barbeau reopened this Jul 30, 2014
@barbeau
Copy link
Member Author

barbeau commented Jul 30, 2014

Looks like nexus.onebusaway.org is down again, but this time isn't responding at all - reported at opentripplanner/OpenTripPlanner#1463

@barbeau barbeau mentioned this issue Jul 30, 2014
Closed
@barbeau
Copy link
Member Author

barbeau commented Aug 4, 2014

To update everyone - there is a temporary fix in place that makes the existing mvn repo hosted at UW functional, so I'm going to close this issue. We're looking at moving this to a virtualized server at USF for long-term stability.

@barbeau barbeau closed this as completed Aug 4, 2014
@barbeau barbeau changed the title Build is broken due to missing maven artifacts Intermittent failures to download maven dependencies Oct 13, 2014
@barbeau
Copy link
Member Author

barbeau commented Oct 13, 2014

Looks like this is an issue again - nexus.onebusaway.org is currently timing out.

Reported by new user on the mailing lists:
https://groups.google.com/forum/#!topic/onebusaway-developers/bagZPozXR68
https://groups.google.com/forum/#!topic/onebusaway-developers/DEl0nZl_8fk

@barbeau barbeau reopened this Oct 13, 2014
@smrose
Copy link
Member

smrose commented Oct 14, 2014

It's back. That service went out when our building lost cooling on 6 October. Today I requested that it be brought back up, and that seems to have succeeded.

@smrose smrose closed this as completed Oct 14, 2014
@barbeau
Copy link
Member Author

barbeau commented Oct 14, 2014

Thanks @smrose!

@slecorne
Copy link

It looks like nexus.onebusaway.org is down again. Got an internal 500 server error while trying to get binary snapshots:
http://nexus.onebusaway.org/service/local/artifact/maven/content?r=public&g=org.onebusaway&a=onebusaway-gtfs-merge-cli&v=1.3.4-SNAPSHOT

@smrose
Copy link
Member

smrose commented Oct 28, 2014

Looks like the file system failed - I can't even log in. It was scheduled,
as announced some time ago, to be retired in less than a week anyway. As
far as I know, no plan to replace it has been forged.

On Tue, Oct 28, 2014 at 7:47 AM, slecorne notifications@github.com wrote:

It looks like nexus.onebusaway.org is down again. Got an internal 500
server error while trying to get binary snapshots:

http://nexus.onebusaway.org/service/local/artifact/maven/content?r=public&g=org.onebusaway&a=onebusaway-gtfs-merge-cli&v=1.3.4-SNAPSHOT

—
Reply to this email directly or view it on GitHub
#93 (comment)
.

S. Morris Rose
technical staff
University of Washington CSE

@alanb2718 alanb2718 reopened this Oct 28, 2014
@alanb2718
Copy link

To provide a bit more detail: nexus.onebusaway.org has been hosted on a really ancient machine at the University of Washington. We've been talking for months if not years about the fact that it could fail at any time and we need to move maven to a different host. To mollify our lab management (who has been bugging us about getting that machine out of the building) we said that we would decommission it Oct 31 at the latest. So it looks like All Hallow's Eve has come a couple of days early, as far as this machine is concerned.

There is a new-ish machine at UW (helsinki.cs.washington.edu) that could host it, or it could be on a new virtual server somewhere else. (The virtual server at USF turned out to be too expensive though.)

@ramunask
Copy link

Why not to place artifacts in central maven repository?

@bdferris
Copy link
Member

We do put some of our libraries in the central maben repository (eg. onebusaway-gtfs) if we think they'll actually be used by others as build dependencies. However, the central repository isn't appropriate for snapshots produced by the CI server or for large release artifacts (the application suite, for example) that aren't really used as build dependencies. Finally, our own repo serves as a nice cache of the dependencies we do need to build OBA (at least, if you are located near Seattle ;)

@barbeau
Copy link
Member Author

barbeau commented Oct 28, 2014

@bdferris I haven't looked at this closely, but when I was having issues setting up a new machine a few months ago it did seem that some of the critical build dependencies for oba-app-modules that aren't OBA-specific (e.g., com.vividsolutions:jts - see this comment) are being pulled off the OBA maven repo, instead of maven central or somewhere else non-OBA. Is this intentional/required as part of the current OBA build structure? If not, we may want to revisit some of the pom.xml files.

@bdferris
Copy link
Member

Maybe rings some bells? Perhaps jts wasn't in the main repo back in the day but is now?

@barbeau
Copy link
Member Author

barbeau commented Oct 28, 2014

That would make sense - Maven Central search is actually timing out for me right now, so I can't see if JTS is up there now :).

On the general topic of OBA artifacts - another option is JCenter. I know Android Gradle plugin artifacts moved there fairly recently, and if the Android team is doing it, it must be the right thing to do ;). Seriously, though, it seems like it is fairly OSS-friendly, although from a quick look I wasn't able to find exact quotas on size for free OSS projects (if there are quotas for OSS projects). I just pinged them to see. According to Bintray, benefits are here. Caveat - I haven't had a chance to look into all the trade-offs between Maven Central vs. JCenter, so this isn't a strong endorsement of going in that direction, just an FYI.

@barbeau
Copy link
Member Author

barbeau commented Oct 29, 2014

From JCenter:

The main limits of a free account are 500MB of storage, maximum of 50MB per file, and 300 REST API calls per day.

@bdferris
Copy link
Member

As a data point, the current size of all org/onebusaway release artifacts is in our existing Nexus repository is 9.2 GB. That includes everything we've ever released.

@barbeau barbeau mentioned this issue Oct 29, 2014
Open
@barbeau
Copy link
Member Author

barbeau commented Oct 29, 2014

Looks like JTS is available via Maven Central - I just opened a new ticket (#103) to reference the artifact there, and review others that might be available in Maven Central.

@sheldonabrown
Copy link
Member

For those critically blocked on this, I have a backup in place now at:

http://developer.onebusaway.org/content/

This will eventually become the new server as soon as it passes testing and we make the appropriate DNS changes.

@barbeau
Copy link
Member Author

barbeau commented Oct 30, 2014

Thanks @sheldonabrown!

@barbeau barbeau mentioned this issue Nov 17, 2014
Closed
@ed-g
Copy link

ed-g commented Jun 23, 2017 •
edited
Loading

Hi everyone,

It looks like both http://developer.onebusaway.org/content/ and http://nexus.onebusaway.org/content/groups/public/ contain no dependancies.

The pom.xml from 1.1.16 which appears to be current stable branch references nexus: https://github.com/OneBusAway/onebusaway-application-modules/tree/onebusaway-application-modules-1.1.16

For me the build does not work:

Downloading: http://nexus.onebusaway.org/content/groups/public/org/onebusaway/onebusaway/1.2.1/onebusaway-1.2.1.pom
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.onebusaway:onebusaway-application-modules:1.1.16: Could not find artifact org.onebusaway:onebusaway:pom:1.2.1 in public.onebusaway.org (http://nexus.onebusaway.org/content/groups/public/) and 'parent.relativePath' points at wrong local POM @ line 3, column 11
 @ 
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.onebusaway:onebusaway-application-modules:1.1.16 (/home/ec2-user/onebusaway-application-modules/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM for org.onebusaway:onebusaway-application-modules:1.1.16: Could not find artifact org.onebusaway:onebusaway:pom:1.2.1 in public.onebusaway.org (http://nexus.onebusaway.org/content/groups/public/) and 'parent.relativePath' points at wrong local POM @ line 3, column 11 -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

Is there an alternate URL where I can grab one bus away dependencies?

Thanks

@sheldonabrown
Copy link
Member

sheldonabrown commented Jun 24, 2017 via email

@sheldonabrown
Copy link
Member

sheldonabrown commented Jun 24, 2017 via email

@ed-g
Copy link

ed-g commented Jun 24, 2017

@sheldonabrown thank you!!!

@sheldonabrown
Copy link
Member

Ok -- I've been working on the configuration, and I've think I've got it back to where it should be.

If it isn't send me the links that break and I'll take a look.

sheldonabrown added a commit that referenced this issue Jan 1, 2024
@sheldonabrown
Merge pull request #93 from camsys/app_mods_log4j2_upgrade
2b2a6b3 
App mods log4j2 upgrade
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@bdferris @alanb2718 @barbeau @sheldonabrown @ed-g @smrose @ramunask @slecorne