Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Geographical CDN issue causing meteor & package install problem #6960

Closed
dksharp opened this issue May 3, 2016 · 297 comments
Closed

Geographical CDN issue causing meteor & package install problem #6960

dksharp opened this issue May 3, 2016 · 297 comments
Assignees

Comments

@dksharp
Copy link

dksharp commented May 3, 2016

To China (Mainland) Users:

You should just try VPN/Shadowsocks or other things to get access to the internet, and set HTTP_PROXY, HTTP_PROXY to use your bridge to the internet.
您最好尝试使用各类工具以访问完整的互联网,并设置 HTTP_PROXYHTTP_PROXY 来让 Meteor 使用您的工具。

Original Issue Content

div@dk:~$ curl https://install.meteor.com/ | sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6675 0 6675 0 0 4772 0 --:--:-- 0:00:01 --:--:-- 4774
Downloading Meteor distribution
### 4.2%
curl: (18) transfer closed with 168048742 bytes remaining to read

    gzip: stdin: unexpected end of file
    tar: Unexpected EOF in archive
    tar: Unexpected EOF in archive
    tar: Error is not recoverable: exiting now
    Installation failed.
@abernix
Copy link
Contributor

abernix commented May 3, 2016

@dksharp Seems like a connectivity problem.

Can you see what happens if you run this command?

Mac OS X
curl -O --progress-bar "https://static-meteor.netdna-ssl.com/packages-bootstrap/1.3.2.4/meteor-bootstrap-os.osx.x86_64.tar.gz"
Linux
curl -O --progress-bar "https://static-meteor.netdna-ssl.com/packages-bootstrap/1.3.2.4/meteor-bootstrap-os.linux.x86_64.tar.gz"

If the same thing happens, will you provide the output of: host static-meteor.netdna-ssl.com
And maybe even a traceroute static-meteor.netdna-ssl.com

@dksharp
Copy link
Author

dksharp commented May 3, 2016

@abernix
Same thing happened.

output of: host static-meteor.netdna-ssl.com-
static-meteor.netdna-ssl.com has address 198.232.125.32;; connection timed out; no servers could be reached

traceroute to static-meteor.netdna-ssl.com (198.232.125.32), 30 hops max, 60 byte packets
1 tata-photon-max.wifi (192.168.1.1) 1.551 ms 1.790 ms 2.140 ms
2 172.29.145.129 (172.29.145.129) 52.426 ms 60.799 ms 67.414 ms
3 172.29.145.140 (172.29.145.140) 67.639 ms 75.511 ms 75.629 ms
4 172.29.145.169 (172.29.145.169) 82.270 ms 82.486 ms 90.231 ms
5 115.113.247.17.static-delhi.vsnl.net.in (115.113.247.17) 89.510 ms 100.620 ms 100.846 ms
6 172.29.250.33 (172.29.250.33) 122.365 ms 117.457 ms 125.681 ms
7 ix-ae-0-100.tcore1.MLV-Mumbai.as6453.net (180.87.38.5) 132.406 ms 83.176 ms 90.256 ms
8 if-ae-9-5.tcore1.WYN-Marseille.as6453.net (80.231.217.17) 205.243 ms if-ae-9-6.tcore1.WYN-Marseille.as6453.net (80.231.217.77) 205.472 ms 212.367 ms
9 if-ae-8-1600.tcore1.PYE-Paris.as6453.net (80.231.217.6) 213.368 ms 213.596 ms 214.214 ms
10 if-ae-2-2.tcore1.PVU-Paris.as6453.net (80.231.154.17) 213.091 ms 212.853 ms 215.080 ms
11 ae12.mpr1.cdg11.us.zip.zayo.com (64.125.14.49) 218.599 ms 213.933 ms 213.805 ms
12 ae4.mpr2.cdg12.fr.zip.zayo.com (64.125.22.166) 243.455 ms 235.928 ms 243.434 ms
13 netdna-gw.cdg.ip4.above.net (94.31.25.206) 177.279 ms 180.537 ms 177.367 ms
14 32-125-232-198.static.unitasglobal.net (198.232.125.32) 190.672 ms 183.886 ms 191.000 ms

PS: I have seen one recent same question, maybe of yesterday, on stackoverflow. same problem.

@dksharp
Copy link
Author

dksharp commented May 3, 2016

I am able to do all other work just fine. installing other things and all. Connection is fine. Only installing meteor is giving me this error.

@abernix
Copy link
Contributor

abernix commented May 3, 2016

@dksharp There were no additional error messages from the curl command? RocketChat/Rocket.Chat#6944 might be related, though I'm not having any problems at all and I'm using the same 198.232.125.32 as you.

But you're right, I've seen other recent similarities #6723 (comment).

@dksharp
Copy link
Author

dksharp commented May 3, 2016

@abernix I have been trying this since Sunday. can't find any solution.

@abernix
Copy link
Contributor

abernix commented May 3, 2016

Ok, new command, old CDN:

Mac OS X
curl -O --progress-bar "https://d3sqy0vbqsdhku.cloudfront.net/packages-bootstrap/1.3.2.4/meteor-bootstrap-os.osx.x86_64.tar.gz"
Linux
curl -O --progress-bar "https://d3sqy0vbqsdhku.cloudfront.net/packages-bootstrap/1.3.2.4/meteor-bootstrap-os.linux.x86_64.tar.gz"

@dksharp, Can you see if that works? (It's a 167MB file, just FYI, so it needs a long, reliable download.)

@dksharp
Copy link
Author

dksharp commented May 3, 2016

@abernix Yeah. It works. Why that standard command is not working :(

@abernix
Copy link
Contributor

abernix commented May 3, 2016

It's a network problem. I can't say for sure why, but it seems to be a problem with the new CDN. It's only happening to some people. Thank you very much for helping to figure it out.

Could you try one more time with this (slightly different) command: (which enables cURL retries and file-resume)

Mac OS X
curl -O -C - --retry 4 --progress-bar "https://static-meteor.netdna-ssl.com/packages-bootstrap/1.3.2.4/meteor-bootstrap-os.osx.x86_64.tar.gz"
Linux
curl -O -C - --retry 4 --progress-bar "https://static-meteor.netdna-ssl.com/packages-bootstrap/1.3.2.4/meteor-bootstrap-os.linux.x86_64.tar.gz"

Thanks again, @dksharp

@dksharp
Copy link
Author

dksharp commented May 3, 2016

Its working. I think new CDN is closing connection abruptly or something like that.
Thank you for helping me out @abernix .

@abernix
Copy link
Contributor

abernix commented May 3, 2016

Ok, that last command works though? If it does, then we can look into changing the install script to use those extra flags. Please report back if it works with the --retry or the -C - (or if it needs both!) 😄

@abernix
Copy link
Contributor

abernix commented May 3, 2016

Btw, while this is not the recommended installation method, you can use that bootstrap tarball by:

tar xfz meteor-bootstrap-os.YOUR_OS_ARCH.tar.gz -C $HOME

Which will install it into your home directory in the .meteor directory (the correct place).

You can then:

  • Add "$HOME/.meteor" to your path,
  • (optional) Install the meteor launcher in /usr/local/bin/ using this command (requires root):
    • sudo cp $HOME/.meteor/$(dirname $(readlink "$HOME/.meteor/meteor"))/scripts/admin/launch-meteor /usr/local/bin/meteor

@dksharp
Copy link
Author

dksharp commented May 3, 2016

The last command worked. I did it using both flags. Will report with using individually.

@glasser
Copy link
Contributor

glasser commented May 3, 2016

Thanks, we're looking into this.

@glasser
Copy link
Contributor

glasser commented May 3, 2016

Folks who are having problems (eg @dksharp): Where in the world are you located? This may be a geographically specific issue with our new CDN.

@abernix
Copy link
Contributor

abernix commented May 3, 2016

I saw India (2x) and Lebanon.

@n1mmy
Copy link
Contributor

n1mmy commented May 3, 2016

I've replicated the issue with a VPN to India. We'll be filing a support ticket with the CDN when we have more details.

In the mean time, we've reverted to the old CDN for the installer.

The meteor package updater (warehouse.meteor.com) is still using the new CDN. We have not yet seen issues with package downloads, but if anyone spots a download issue with packages (as opposed to installers) please let us know!

Thanks again, @abernix, for your help here. Much appreciated.

@allpratik
Copy link

allpratik commented May 3, 2016

Yes, guys I'm facing this issue in India from past 3+ days. Though, I did installed meteor manually but it was painful as compared to our recommended process. Also, becuase of this...now everytime I had to use sudo with meteor command.

@glasser
Copy link
Contributor

glasser commented May 3, 2016

@allpratik When did you see this? We reverted an hour or two ago, so if you're still seeing problems then there's a surprise.

@glasser
Copy link
Contributor

glasser commented May 3, 2016

Or do you mean you're seeing the issue downloading packages?

@allpratik
Copy link

@glasser No I didn't tried packages. Also, I tested it before 10 hrs. So, let me check current situation and also check packages as well.

@n1mmy n1mmy self-assigned this May 3, 2016
@n1mmy
Copy link
Contributor

n1mmy commented May 3, 2016

I tested downloading multiple files over the VPN to India. Curious results. Here's what I've found so far:

  • warehouse.meteor.com downloads via meteor cli: GOOD. (tested by running meteor --release 1.3.2.2 --help and downloading a new tool, for 3 different releases)
  • curl -v -o /dev/null https://static-meteor.netdna-ssl.com/packages-bootstrap/1.3.2.1/meteor-bootstrap-os.linux.x86_64.tar.gz: BAD (fails consistently)
  • curl -v -o /dev/null https://warehouse.meteor.com/builds/EXSxwGqYjjJKh3WMJ/1459125225532/3AAHkWZLi8/babel-compiler-6.5.2-os+web.browser+web.cordova.tgz (small file): GOOD
  • curl -v -o /dev/null https://warehouse.meteor.com/builds/EXSxwGqYjjJKh3WMJ/1459133033542/egJ3SEgtYM/meteor-tool-1.3.0_3-os.linux.x86_64+web.browser+web.cordova.tgz (big file): BAD (fails consistently)

So it seems the error only occurs with big files, via curl not the meteor tool, from some places (only tested India so far).

@n1mmy
Copy link
Contributor

n1mmy commented May 4, 2016

If anyone else who saw this issue could please post their IP address and the output of traceroute static-meteor.netdna-ssl.com that would be very useful for us to debug.

@n1mmy
Copy link
Contributor

n1mmy commented May 4, 2016

And if you are not comfortable posting your IP publicly, feel free to email me privately at nick@meteor.com.

@abernix
Copy link
Contributor

abernix commented May 4, 2016

@shaileshtheroyal You also had problems installing/downloading, I believe. Could you provide your information as @n1mmy posted above?

@n1mmy Is it possible the Windows installer is/was accessing the new CDN already as well? If so, maybe reach out to the two people in RocketChat/Rocket.Chat#6955 for their traceroute...err...tracert.

@saltslag
Copy link

saltslag commented May 4, 2016

@n1mmy I've been having the same problem in Belgium last couple of days. I can confirm it's due to bad internet connection but I don't think it's because of curl. When trying wget, the same problem occurred.

Network at my student house is pretty bad. Downloading over 4G did the trick for me.

@gothicn
Copy link

gothicn commented May 4, 2016

Sounds like my problem...

MacOS standart installation doesn't go over 27%
traceroute returns a stack of ***

@n1mmy
Copy link
Contributor

n1mmy commented May 4, 2016

@abernix Yup, windows is the same CDN as the installer. Updated the windows ticket too. Thanks.

@gothicn @titmastercrunch Not sure if you folks have the same issue, but I'd really appreciate it if you could try something for me. Can you please run each of these commands:

curl https://static-meteor.netdna-ssl.com/packages-bootstrap/1.3.2.4/meteor-bootstrap-os.linux.x86_64.tar.gz -o /dev/null

curl https://d3sqy0vbqsdhku.cloudfront.net/packages-bootstrap/1.3.2.4/meteor-bootstrap-os.osx.x86_64.tar.gz -o /dev/null

2 or 3 times each and report back the results?

@gothicn
Copy link

gothicn commented May 6, 2016

Everything fine. No errors, download finishes 100% successfully. MacOS el capitan
It seems i got another type problem... It always stops at 27-30% with curl |sh
And also freezes at 27/60% of manual installation pre-build download step, finishing with SSL error

Should i create a separate issue? I didn't find any solutions.
Few men with(prolly) the same issue had it resolved by a miracle.

@mitar
Copy link
Contributor

mitar commented Sep 1, 2016

But when Meteor is downloading packages, it is hitting up CDN with multiple requests in serial, not only one curl one, no? And then there are multiple ones in parallel.

Maybe you can use the Apache benchmarking tool ab to test it?

@MalcolmGC
Copy link

MalcolmGC commented Sep 1, 2016

I appreciate all the good information. Here is what I have done after I think I understood how to avoid the meteor-tool issue: taking a long time, apparently randomly, when I enter > meteor to run a project.
I am in an online course, and each week they give us a new project beginning. When I put it into the appropriate folder and run it (>meteor) it then takes a long time or even hangs running overnight. I am using Windows 7, but I do not think this is the real issue (I could be wrong). I have searched (using "File Finder") for meteor-tool and found Giga-bytes of different versions (e.g., 1.4.0-1).
Here is what I did (finally). Uninstall Meteor. Delete all .meteor folders on my computer (Not easy on Windows -- see other, but you may have to move to C:\ first or other such ways to get around the "long file name" problem.)
Now, start over. Install Meteor. Run (from a projects folder) > Meteor create myTest to create a myTest project. >cd myTest and then > Meteor to run the project. You can open localhost:3000 to verify that everything is good to here.
Now, go to myTest.meteor and open "release" in a text editor. Mine today shows METEOR@1.4.1.1
When I look in C:\Documents and Settings\ ... \AppData\Local.meteor\packages\meteor-tool, I see a folder 1.4.4_1 and in ... \AppData\Local\PaxHeader.meteor\packages\meteor-tool
C:\Documents and Settings\Currie Win7 Desk\AppData\Local.meteor\packages\meteor-tool, I also see a folder 1.4.4_1 .
Good, now when I go to the provided project, I go to provided_project.meteor and open "release" in a text editor I see something like this: METEOR@1.3.0.1 If I leave this, and run meteor, I have the problem which is being addressed in these posts. However, if I change this to METEOR@1.4.1.1 (the same as I had in the installation -- notice slight change in how it is presented in this file and in the computer folder: 1.4.4.1 vs 1.4.4_1 (which I do not understand, but it is what it is).
*** So again, change the project "release" to 'agree' with the installation folder. ***
That is the secret, apparently. Then when I start the project, it just runs!
Hope this helps you with this issue
Malcolm

@tmeasday
Copy link
Contributor

tmeasday commented Sep 1, 2016

@mitar in your CI scripts is the tool downloading older versions? Do you install 1.4.1.1, then run meteor in an older app? Or is the problem the install in the first place?

A workaround (if it is the tool being slow but not curl), would be install the version that you need to begin with with curl https://install.meteor.com?release=X.Y.Z | sh in the CI script.

@mitar
Copy link
Contributor

mitar commented Sep 2, 2016

Hm, you are right. I have both devel branch and latest stable Meteor. See here. Do you think having both version on the system could make things break?

@tmeasday
Copy link
Contributor

tmeasday commented Sep 2, 2016

@mitar, not necessarily. But am I right in saying that your test script is just running meteor test-packages ./ inside each package directory (or something equivalent)? I can't see why it would need to download older versions of the tool. Can you link me to a timing out build?

@mitar
Copy link
Contributor

mitar commented Sep 2, 2016

I linked it above: https://circleci.com/gh/meteor/blaze/67

@mitar
Copy link
Contributor

mitar commented Sep 2, 2016

And no, it should not download latest version of the tool.

@tmeasday
Copy link
Contributor

tmeasday commented Sep 2, 2016

@mitar it seems to have hung at "Building plugin lintJshint in packa...", which doesn't seem to be anything to do with downloading the tool. No idea what that could be about.

I think we've seen similar timeout errors of late with meteor/meteor on circle too. I'm not sure if it's something meteor is doing differently (not sure what it could be TBH) or if it's a problem with Circle itself.

@mitar
Copy link
Contributor

mitar commented Sep 2, 2016

Hm, maybe the log is cutoff and this is not the latest thing?

@mitar
Copy link
Contributor

mitar commented Sep 2, 2016

CircleCI failure:

Updating package catalog
DDP disconnected while connection in progress
This command requires an up-to-date package catalog.  Exiting.

/tmp/meteor/meteor --get-ready returned exit code 1

Action failed: /tmp/meteor/meteor --get-ready

@ameensom
Copy link

You are hitting the MaxCDN Paris Datacenter
2.88.120.221
Riyadh, Saudi Arabia
I'm getting an error when creating an app using meteor create appname
it tries to download the packages but i get this errors :
=> Errors while creating your project

While downloading autoupdate@1.2.11...:
error: connect EHOSTDOWN 185.18.207.126:443 - Local (0.0.0.0:0)

While downloading caching-compiler@1.0.6...:
error: connect EHOSTDOWN 185.18.207.126:443 - Local (0.0.0.0:0)

While downloading ddp-client@1.2.9...:
error: connect EHOSTDOWN 185.18.207.126:443 - Local (0.0.0.0:0)

While downloading ddp-server@1.2.10...:
error: connect EHOSTDOWN 185.18.207.126:443 - Local (0.0.0.0:0)

While downloading http@1.1.8...:
error: connect EHOSTDOWN 185.18.207.126:443 - Local (0.0.0.0:0)

While downloading templating@1.1.14...:
error: connect EHOSTDOWN 185.18.207.126:443 - Local (0.0.0.0:0)

@n1mmy @MalcolmGC

@LongLiveCHIEF
Copy link

I've been trying to install meteor, over the course of several months now, attempting to contribute Markdown Support to Rocket.Chat. You can see there, that I've been working on this since Jul 22. All that time, I have tried to install on several machines, (latest 2 releases of 10.x, Several flavors of Ubuntu 16, and once, even Windows 10 out of desperation) and even tried several different networks, just to ensure it wasn't my networking situation.

I've even used corkscrew to set up tunneling for me on a migratory bastion server that I've moved around different regions using DO droplets. In each case, it's always the same:

I run:

curl https://install.meteor.com/ | sh

And in each case, I get:

curl: (7) Failed to connect to meteorinstall-4168.kxcdn.com port 443: Connection timed out
Retrying download in 5 seconds...

// repeat 9 more times

@tcastelli
Copy link
Contributor

tcastelli commented Nov 3, 2016

I started having problems to download meteor packages since one week ago and it has become worse and worse, not sure if anyone still experiencing the same (meteor per se is downloaded but packages fail):

maxcdn

You are hitting the MaxCDN Paris Datacenter

amazonwsip

84.76.115.221

tracert

static-meteor.netdna-ssl.com [94.31.29.248]

1 2 ms 3 ms 2 ms LIVEBOX [192.168.1.1]
2 26 ms 25 ms 25 ms 84.76.115.254
3 26 ms 29 ms 25 ms 10.255.205.113
4 23 ms 24 ms 24 ms 10.255.204.41
5 29 ms 27 ms 27 ms 10.34.212.157
6 * * * Tiempo de espera agotado para esta solicitud.
7 * * * Tiempo de espera agotado para esta solicitud.
8 * * * Tiempo de espera agotado para esta solicitud.
9 47 ms 43 ms 43 ms 10.34.34.45
10 44 ms 43 ms 43 ms bundle-ether101.madtr3.Madrid.opentransit.net [193.251.249.1]
11 43 ms 44 ms 45 ms be3012.ccr22.mad05.atlas.cogentco.com [130.117.14.137]
12 55 ms 49 ms 48 ms be2293.ccr22.bio02.atlas.cogentco.com [130.117.50.25]
13 61 ms 61 ms 61 ms be2423.ccr42.par01.atlas.cogentco.com [130.117.50.77]
14 62 ms 60 ms 61 ms be2108.agr22.par01.atlas.cogentco.com [130.117.50.133]
15 61 ms 62 ms 62 ms be2947.rcr12.b019498-0.par01.atlas.cogentco.com [154.25.7.105]
16 63 ms 64 ms 64 ms netdna-gw.cdg.ip4.cogentco.com [149.6.114.82]
17 63 ms 63 ms 63 ms 94.31.29.248.IPYX-077437-ZYO.above.net [94.31.29.248]

error

Errors prevented startup:

While downloading alanning:roles@1.2.15...:
error: connect ETIMEDOUT 54.192.62.173:443

While downloading aldeed:autoform@5.8.1...:
error: connect ETIMEDOUT 54.192.62.173:443

While downloading aldeed:collection2@2.10.0...:
error: connect ETIMEDOUT 54.192.62.173:443

While downloading aldeed:collection2-core@1.2.0...:
error: connect ETIMEDOUT 54.192.62.173:443

While downloading aldeed:schema-deny@1.1.0...:
error: connect ETIMEDOUT 54.192.62.173:443

While downloading aldeed:schema-index@1.1.1...:
error: connect ETIMEDOUT 54.192.62.173:443

While downloading aldeed:simple-schema@1.5.3...:
error: connect ETIMEDOUT 54.192.62.173:443

While downloading aldeed:template-extension@4.0.0...:
error: connect ETIMEDOUT 54.192.62.173:443

While downloading amplify@1.0.0...:
error: connect ETIMEDOUT 54.192.62.173:443

..etc

EDIT: After a long number of retries and waiting for some time it finally worked 👍

@niwsa
Copy link

niwsa commented Nov 4, 2016

@abernix @n1mmy Can something like https://unpkg.com/ help solve the issue for once and for all.

@abernix
Copy link
Contributor

abernix commented Nov 4, 2016

I think we need a participant vote on this, so please bear with this wall of text and vote at the end because I was about to close this issue about an hour before @tcastelli posted his most recent update. I felt like this was resolved best-as-possible, for a number of reasons:

  1. First, these are my (shortened) comments from about a week ago on another thread about an actual, identified problem with the current package server CDN provider which was occurring. Upon fixing the problem I saw improvement for many users (determined by a noticeable decrease in install-related issue comments here in GitHub Issues across many issues, and success reported on IRC, Slack, emotes and more):

    Surprisingly, the CDN was actually just behaving poorly – for example, I just straight-up could not wget the entire meteor-tool through the CDN without it (remotely) closing the connection. I used to be able to, and then it just stopped working

    I just reached out to MaxCDN Support over chat and worked with their support team for almost 45 minutes and they reproduced and corrected an issue on their end. For the first time in the last two weeks, I am able to fully download Meteor without using any work-arounds.

  2. The Meteor Installer now supports retrying on install thanks to Pagination of LDAP users in sync RocketChat/Rocket.Chat#7348

  3. The Meteor package installer now supports retrying thanks to LiveChat Agent Presence RocketChat/Rocket.Chat#7399

  4. And most recently, (about a week ago) Meteor 1.4.1.3 and Meteor 1.4.2 more intelligently deal with closed network connections (and trigger the above retry) thanks to 545493b.

  5. This issue was actually IDLE for an entire month. I don't feel that would have happened without the above changes and I think this shows the problem is fixed for many users.

  6. This issue has grown a bit out of control and it's hard to even be sure of specific issues anymore – for example, package downloads, NPM downloads, installer downloads – these all come from different servers and completely different networks! Any updates or +1s are probably involving dozens and dozens of people that aren't even having problems (or the same problem) anymore

  7. And related to the previous bullet, I think that @tcastelli's problem might be a specific problem with package-downloading/retrying which only affects Windows users. I've seen it myself on Windows from a connection where I have zero Meteor problems from Mac/Unix boxes. (I believe it happens when too many packages are attempted to be downloaded at once, versus a more delicate 2-at-a-time approach.)

Network problems may never (won't) go away – but Meteor can get better at dealing with them, as has tried with the above solutions.

@niwsa, I'm not sure that replacing one CDN with another CDN is the solution (nor do I think it would be a "once and for all"). Unpkg is really just another NPM CDN and using it, even right now, wouldn't solve this problem because Meteor is more than just NPM – its packages are not all NPM and are in their own atmosphere, well, literally.

It could be argued that the best-of-the-best CDN would be the most ideal for Atmosphere packages, but even if a particular CDN performs well for some users, it won't for others – a huge number of Meteor users are in places where network conditions just actually are bad – for example, China where the wall makes it very difficult for anything to work properly – NPM or otherwise! There is no putting CDN problems to rest and NPM has fought with this problem for a long time - Meteor has an additional angle of needing packages from two sources: its own and NPM.

Let's vote, using reactions and comments

  • I'll consider a 👍 reaction on this comment as a vote to "close" this issue. This should indicate that you think the CDN issues are resolved or improved enough. Of course, specific issues with specific conditions like @tcastelli's (package download+Windows+ETIMEDOUT) are always welcome in new issues.
  • If you have any other feeling other than closing this issue, please post specifically what you are still having problems with (e.g. package/installer/NPM), include actual error messages (e.g. transfer closed, connection interrupted, ECONNRESET, ETIMEOUT, etc), platforms (Windows, Mac), etc.

Based on the results, I will create new issues for specific problems, create action items, and we can work together to resolve those with the users who are affected by them.

My vote is still to close.

@tcastelli
Copy link
Contributor

tcastelli commented Nov 4, 2016

I also think there's always gonna be networks errors and in general I think the new measures in latest versions are normally enough. However, like what happened with METEOR_WAREHOUSE_URLBASE, I think that another env variable could be used for packages url (or not sure if that exists already and we can use the trick to change it like with warehouse). Also some documentation about those env variables would be great (with possible url alternatives when experiencing problems).

Regarding opening more specific issues (Windows + multiple packages at the same time), I also think is a good idea, I just didn't know it was something related to Windows only. @abernix Should I open an issue for that or is that already tracked somewhere else?

@abernix
Copy link
Contributor

abernix commented Nov 4, 2016

@tcastelli METEOR_WAREHOUSE_URLBASE is the environment variable for the package server which serves Atmosphere packages and meteor tool updates (which is actually just an Atmosphere package).

My request to you, for testing purposes, would be to use the Windows workaround suggested in this comment by setting METEOR_WAREHOUSE_URLBASE to the cloudfront URL. If you still have the problem you reported above it shows that this problem exists with either a) your own connection (I'm not saying this is the case); or b) completely separate, isolated networks/CDNs, but ultimately, a Meteor on Windows problem exists with package downloading networking code preventing it from being more graceful (namely retrying, failing and being more informative). If the problem goes away with the cloudfront URL, well, then it definitely indicates a CDN deficiency, but the networking code still needs some work. Either way, I think a "Windows package installation ETIMEDOUT error" issue for that would be a wise idea and I'd encourage anyone with those same circumstances to jump into that issue (and off of this one).

(Also, I quasi-agree on the documentation aspect, but that it should be a last resort as if we can get everything working properly it will take care of itself. I dislike the idea of users having to make these "tweaks", which tend to sit around forever, and think it would be preferred if we could intelligently make those decisions for them.)

@tcastelli
Copy link
Contributor

Thanks for the clarification about warehouse, I tried indeed to change it to cloudfront when I was facing the problem, and it dind't help at that time, so it doesn't seem something related to the CDN then. I have opened a new issue RocketChat/Rocket.Chat#8013 for Windows specifics.

In terms of documentation I think it won't hurt to have it well documented (especially if later on private servers area allowed), but I agree that automatically trying to fix the problem would be ideal, so maybe internally an array of WAREHOUSE_URLBASE urls could be stored in the build tool, and try to fetch from a different one if the first fails.

@mitar
Copy link
Contributor

mitar commented Nov 4, 2016

It could be argued that the best-of-the-best CDN would be the most ideal for Atmosphere packages, but even if a particular CDN performs well for some users, it won't for others

Hm, but the previous CDN worker very well and none of those issues were present there. The long list of fixes you had to introduce now for retrying and all other stuff were simply not present there and there were no such long issues like this one with that CDN. So we do not have to talk about "best-of-the-best CDN" in abstract terms, we can just move to a previous CDN?

@abernix
Copy link
Contributor

abernix commented Nov 4, 2016

I don't think you can say they were not present, just that you did not experience them.

There were absolutely problems with the old CDN as well – some of the problems that the community was hit with full force with the change to the new CDN (which by the way, again had an identified problem) were in-fact long-standing problems which may have been skipped over in the past. I think the improvements to the tool all make a lot of sense and most of them should have been there before. These have all made the tool more resilient for folks with sub-par network conditions, and the bottom line is it seems to be working now.

Ultimately, per my comments here I'm guessing this comes down to the graciousness (emphasis on the emphasis) of MaxCDN and that makes a lot of sense. Amazon is probably less gracious.

@laosb
Copy link
Contributor

laosb commented Nov 5, 2016

Though China users would still suffer network problems, it seems that the CDN works well for most developers. Thanks @abernix and MDG ones! I guess we can close this.

PS: I had already edited the issue content of this issue for China (Mainland) users. It should work.

@cyclops24
Copy link

My issue also fixed and I can install Meteor and packages successfully. Thanks MDG.

@abernix
Copy link
Contributor

abernix commented Nov 18, 2016

Per my comment above (and because this seems to actually be solved), I'm closing this and locking further conversation on it.

I know this issue hit users hard. Thank you to the large number of participants who reported problems, submitted traceroutes, or otherwise provided improvements to the meteor tooling. Hopefully this will be far less of an issue in the future.

@abernix abernix closed this as completed Nov 18, 2016
@meteor meteor locked and limited conversation to collaborators Nov 18, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests