gem push fails with Broken Pipe IOError

[AndyObtiva]

Environment Information

Provide at least:

• JRuby version (jruby -v) and command line (flags, JRUBY_OPTS, etc): 9.3.0.0
• Operating system and platform (e.g. uname -a): Darwin Andys-MacBook-Pro.local 20.3.0 Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64 x86_64

Other relevant info you may wish to add:

• gem --version returns 3.2.28

Expected Behavior

Running gem push path_to_gem.gem should result in releasing the gem to RubyGems.org successfully.

This works in jruby-9.2.19.0

Actual Behavior

In jruby-9.3.0.0, I get a broken pipe IOError when running gem push path_to_gem.gem:

% gem push pkg/glimmer-dsl-swt-4.21.0.1.gem 
Pushing gem to https://rubygems.org...
ERROR:  While executing gem ... (IOError)
    Broken pipe

It works fine in ruby-3.0.2 and jruby-9.2.19.0. My Internet works well and the same in all cases.

[headius]

Could you try to run this with more debugging output, via some combination of --verbose and --backtrace and --debug?

I'm trying to figure out if there's a dummy RG server somewhere I could use to test this. If you can get more debugging output we might have enough information to narrow the scope of this issue.

[AndyObtiva]

OK, I just did and here is what I got:

% gem push --verbose --backtrace --debug pkg/glimmer-dsl-swt-4.21.0.2.gem 
NOTE:  Debugging mode prints all exceptions even when rescued
Exception `OptionParser::InvalidOption' at /Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/optparse.rb:1769 - invalid option: no-document
Exception `OptionParser::InvalidOption' at /Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/optparse.rb:1769 - invalid option: --no-document
Pushing gem to https://rubygems.org...
POST https://rubygems.org/api/v1/gems
Exception `IOError' at org/jruby/ext/openssl/SSLSocket.java:941 - Broken pipe
Exception `IOError' at org/jruby/ext/openssl/SSLSocket.java:941 - Broken pipe
Exception `IOError' at org/jruby/ext/openssl/SSLSocket.java:941 - Broken pipe
ERROR:  While executing gem ... (IOError)
    Broken pipe
	org/jruby/ext/openssl/SSLSocket.java:941:in `syswrite_nonblock'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/gems/shared/gems/jruby-openssl-0.10.7-java/lib/jopenssl23/openssl/buffering.rb:383:in `write_nonblock'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:296:in `block in write0'
	org/jruby/RubyArray.java:1865:in `each'
	org/jruby/RubyEnumerable.java:1143:in `each_with_index'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:275:in `write0'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:248:in `block in write'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:265:in `writing'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:247:in `write'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http/generic_request.rb:189:in `send_request_with_body'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http/generic_request.rb:123:in `exec'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http.rb:1518:in `block in transport_request'
	org/jruby/RubyKernel.java:1237:in `catch'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http.rb:1517:in `transport_request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http.rb:1490:in `request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/request.rb:220:in `perform_request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/request.rb:152:in `fetch'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/remote_fetcher.rb:309:in `request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/gemcutter_utilities.rb:234:in `request_with_otp'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/gemcutter_utilities.rb:105:in `rubygems_api_request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/commands/push_command.rb:89:in `send_push_request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/commands/push_command.rb:81:in `send_gem'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/commands/push_command.rb:66:in `execute'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/command.rb:323:in `invoke_with_build_args'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/command_manager.rb:178:in `process_args'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/command_manager.rb:147:in `run'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/gem_runner.rb:53:in `run'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/bin/gem:15:in `<main>'
Exception `Gem::SystemExitException' at /Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/user_interaction.rb:365 - Exiting RubyGems with exit_code 1

I deleted my ~/.gemrc, which had the --no-document option and ran the gem push command again, getting this output:

 % gem push --verbose --backtrace --debug pkg/glimmer-dsl-swt-4.21.0.2.gem
NOTE:  Debugging mode prints all exceptions even when rescued
Pushing gem to https://rubygems.org...
POST https://rubygems.org/api/v1/gems
Exception `IOError' at org/jruby/ext/openssl/SSLSocket.java:941 - Broken pipe
Exception `IOError' at org/jruby/ext/openssl/SSLSocket.java:941 - Broken pipe
Exception `IOError' at org/jruby/ext/openssl/SSLSocket.java:941 - Broken pipe
ERROR:  While executing gem ... (IOError)
    Broken pipe
	org/jruby/ext/openssl/SSLSocket.java:941:in `syswrite_nonblock'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/gems/shared/gems/jruby-openssl-0.10.7-java/lib/jopenssl23/openssl/buffering.rb:383:in `write_nonblock'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:289:in `block in write0'
	org/jruby/RubyArray.java:1865:in `each'
	org/jruby/RubyEnumerable.java:1143:in `each_with_index'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:275:in `write0'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:248:in `block in write'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:265:in `writing'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/protocol.rb:247:in `write'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http/generic_request.rb:189:in `send_request_with_body'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http/generic_request.rb:123:in `exec'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http.rb:1518:in `block in transport_request'
	org/jruby/RubyKernel.java:1237:in `catch'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http.rb:1517:in `transport_request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/net/http.rb:1490:in `request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/request.rb:220:in `perform_request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/request.rb:152:in `fetch'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/remote_fetcher.rb:309:in `request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/gemcutter_utilities.rb:234:in `request_with_otp'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/gemcutter_utilities.rb:105:in `rubygems_api_request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/commands/push_command.rb:89:in `send_push_request'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/commands/push_command.rb:81:in `send_gem'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/commands/push_command.rb:66:in `execute'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/command.rb:323:in `invoke_with_build_args'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/command_manager.rb:178:in `process_args'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/command_manager.rb:147:in `run'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/gem_runner.rb:53:in `run'
	/Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/bin/gem:15:in `<main>'
Exception `Gem::SystemExitException' at /Users/andymaleh/.rvm/rubies/jruby-9.3.0.0/lib/ruby/stdlib/rubygems/user_interaction.rb:365 - Exiting RubyGems with exit_code 1

[headius]

On master, I was able to push a gem ok (Java 11, macOS 10.15). I will try with released 9.3 and see if I can repro.

[headius]

Oh and thank you for the extra logging output! Seems like this ties the issue back to jruby-openssl. However, I think we shipped the same jruby-openssl in 9.2.19.0 so there's more to the story.

I was able to push a gem with the 9.3.0.0 release (installed via rvm) so we need to dig deeper on why this is failing for you.

[] ~/projects/weakling $ jruby -v
jruby 9.3.0.0 (2.6.8) 2021-09-17 85c20e780f OpenJDK 64-Bit Server VM 25.222-b10 on 1.8.0_222-b10 +jit [darwin-x86_64]

[] ~/projects/weakling $ gem -v
3.2.14

@kares Any thoughts here?

[AndyObtiva]

Thanks for giving it a try.

Sorry, I forgot to provide the full version line, which indicates I am using JDK 16.

% jruby -v
jruby 9.3.0.0 (2.6.8) 2021-09-17 85c20e780f Java HotSpot(TM) 64-Bit Server VM 16.0.2+7-67 on 16.0.2+7-67 +jit [darwin-x86_64]

% gem -v
3.2.28

% rvm --version
rvm 1.29.12 (latest) by Michal Papis, Piotr Kuczynski, Wayne E. Seguin [https://rvm.io]

% uname -a
Darwin Andys-MacBook-Pro.local 20.6.0 Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64 x86_64

I am using MacOS Big Sur 11.6

Perhaps you would get different results with JDK 16?

[headius]

I will give that a shot. I also have a macOS 11 system here that I'll try to fire up tomorrow.

[headius]

Pushing on Java 16 appears to work fine for me locally, on Catalina.

[] ~/projects/weakling $ gem push weakling-0.0.5.pre2-java.gem 
Pushing gem to https://rubygems.org...
You have enabled multi-factor authentication. Please enter OTP code.
Code:   433294
Successfully registered gem: weakling (0.0.5.pre2-java)

[] ~/projects/weakling $ java -version
openjdk version "16-ea" 2021-03-16
OpenJDK Runtime Environment (build 16-ea+29-2091)
OpenJDK 64-Bit Server VM (build 16-ea+29-2091, mixed mode, sharing)

[enebo]

I just tried with Fedora Core 34 using both Java 8 and Java 16 and this also works for me. Just thinking outloud perhaps some SSL settings is different for your env than ours and this is tripping over something?

[AndyObtiva]

Thank you for testing with Java 16.

Please keep in mind that gem pushing works for me on the same machine setup with earlier versions of JRuby and with CRuby as well.

Also, did you notice the linkage from this issue by michaelklishin? ruby-amqp/march_hare#156

It seems he encountered the same issue. He also had to resort to CRuby to push his gem.

[headius]

Right, I believe you, I'm just trying to find a way to reproduce it here so we can fix it. I will try a few more things.

[headius]

This is an x86_64 MBP, yes?

[headius]

I attempted to push with Java 16 (x86_64) on a macOS 11.5 M1 machine and it also pushed ok there.

Need to figure out how to get more information from your system. Have we tried all the relevant flags at this point?

[headius]

@michaelklishin If you are able to reproduce we could use another data point. JDK version? OS version? Hardware?

[AndyObtiva]

“This is an x86_64 MBP, yes?”

Yes, correct.

[michaelklishin]

I ran into this with JDK 13.0.2 on MacOS 11.6 (Intel hardware).