Module Bio::Command
In: lib/bio/command.rb

Bio::Command

Bio::Command is a collection of useful methods for execution of external commands or web applications. Any wrapper class for applications shall use this class.

Library internal use only. Users should not directly use it.

Methods

Classes and Modules

Class Bio::Command::Tmpdir

Constants

UNSAFE_CHARS_UNIX = /[^A-Za-z0-9\_\-\.\:\,\/\@\x1b\x80-\xfe]/n
QUOTE_CHARS_WINDOWS = /[^A-Za-z0-9\_\-\.\:\,\/\@\\]/n
UNESCAPABLE_CHARS = /[\x00-\x08\x10-\x1a\x1c-\x1f\x7f\xff]/n

Public Instance methods

Executes the program. Automatically select popen for Ruby 1.9 or Windows environment and fork for the others. A block must be given. An IO object is passed to the block.

Available options:

  :chdir => "path" : changes working directory to the specified path.

Arguments:

  • (required) cmd: Array containing String objects
  • (optional) options: Hash
Returns:(undefined)

[Source]

     # File lib/bio/command.rb, line 196
196:   def call_command(cmd, options = {}, &block) #:yields: io
197:     if RUBY_VERSION >= "1.9.0" then
198:       return call_command_popen(cmd, options, &block)
199:     elsif no_fork? then
200:       call_command_popen(cmd, options, &block)
201:     else
202:       begin
203:         call_command_fork(cmd, options, &block)
204:       rescue NotImplementedError
205:         # fork(2) not implemented
206:         @@no_fork = true
207:         call_command_popen(cmd, options, &block)
208:       end
209:     end
210:   end

This method is internally called from the call_command method. In normal case, use call_command, and do not call this method directly.

Executes the program via fork (by using IO.popen("-")) and exec. A block must be given. An IO object is passed to the block.

See the document of call_command for available options.

Note for Ruby 1.8: In Ruby 1.8, from the view point of security, this method is recommended rather than call_command_popen. However, this method might have problems with multi-threads.

Note for Ruby 1.9: In Ruby 1.9, this method can not be used, because Thread.critical is removed. In Ruby 1.9, call_command_popen is safe and robust enough, and is the recommended way, because IO.popen is improved to get a command-line as an array without calling shell.


Arguments:

  • (required) cmd: Array containing String objects
  • (optional) options: Hash
Returns:(undefined)

[Source]

     # File lib/bio/command.rb, line 373
373:   def call_command_fork(cmd, options = {})
374:     dir = options[:chdir]
375:     cmd = safe_command_line_array(cmd)
376:     begin
377:     tc, Thread.critical, flag0, flag1 = Thread.critical, true, true, true
378:     IO.popen("-", "r+") do |io|
379:       if io then
380:         # parent
381:         flag0, Thread.critical, flag1 = false, tc, false
382:         yield io
383:       else
384:         # child
385:         Thread.critical = true # for safety, though already true
386:         GC.disable
387:         # chdir to options[:chdir] if available
388:         begin
389:           Dir.chdir(dir) if dir
390:         rescue Exception
391:           Process.exit!(1)
392:         end
393:         # executing the command
394:         begin
395:           Kernel.exec(*cmd)
396:         rescue Errno::ENOENT, Errno::EACCES
397:           Process.exit!(127)
398:         rescue Exception
399:         end
400:         Process.exit!(1)
401:       end
402:     end
403:     ensure
404:       # When IO.popen("-") raises error, Thread.critical will be set here.
405:       Thread.critical = tc if flag0 or flag1
406:       #warn 'Thread.critical might have wrong value.' if flag0 != flag1
407:     end
408:   end

Executes the program via Open3.popen3 A block must be given. IO objects are passed to the block.

You would use this method only when you really need to get stderr.


Arguments:

  • (required) cmd: Array containing String objects
Returns:(undefined)

[Source]

     # File lib/bio/command.rb, line 419
419:   def call_command_open3(cmd)
420:     cmd = safe_command_line_array(cmd)
421:     Open3.popen3(*cmd) do |pin, pout, perr|
422:       yield pin, pout, perr
423:     end
424:   end

This method is internally called from the call_command method. In normal case, use call_command, and do not call this method directly.

Executes the program via IO.popen for OS which doesn‘t support fork. A block must be given. An IO object is passed to the block.

See the document of call_command for available options.

Note for Ruby 1.8: In Ruby 1.8, although shell unsafe characters are escaped. If inescapable characters exists, it raises RuntimeError. So, call_command_fork is normally recommended.

Note for Ruby 1.9: In Ruby 1.9, call_command_popen is safe and robust enough, and is the recommended way, because IO.popen is improved to get a command-line as an array without calling shell.


Arguments:

  • (required) cmd: Array containing String objects
  • (optional) options: Hash
Returns:(undefined)

[Source]

     # File lib/bio/command.rb, line 235
235:   def call_command_popen(cmd, options = {}, &block)
236:     if RUBY_VERSION >= "1.9.0" then
237:       if RUBY_ENGINE == 'jruby' then
238:         _call_command_popen_jruby19(cmd, options, &block)
239:       else
240:         _call_command_popen_ruby19(cmd, options, &block)
241:       end
242:     else
243:       _call_command_popen_ruby18(cmd, options, &block)
244:     end
245:   end

Escape special characters in command line string.


Arguments:

Returns:String object

[Source]

     # File lib/bio/command.rb, line 121
121:   def escape_shell(str)
122:     if windows_platform? then
123:       escape_shell_windows(str)
124:     else
125:       escape_shell_unix(str)
126:     end
127:   end

Escape special characters in command line string for UNIX shells.


Arguments:

Returns:String object

[Source]

     # File lib/bio/command.rb, line 110
110:   def escape_shell_unix(str)
111:     str = str.to_s
112:     raise 'cannot escape control characters' if UNESCAPABLE_CHARS =~ str
113:     str.gsub(UNSAFE_CHARS_UNIX) { |x| "\\#{x}" }
114:   end

Escape special characters in command line string for cmd.exe on Windows.


Arguments:

Returns:String object

[Source]

     # File lib/bio/command.rb, line 95
 95:   def escape_shell_windows(str)
 96:     str = str.to_s
 97:     raise 'cannot escape control characters' if UNESCAPABLE_CHARS =~ str
 98:     if QUOTE_CHARS_WINDOWS =~ str then
 99:       '"' + str.gsub(/\"/, '""') + '"'
100:     else
101:       String.new(str)
102:     end
103:   end

Same as:

 http = Net::HTTP.new(...); http.post_form(path, params)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set. In addition, header can be set. (Note that Content-Type and Content-Length are automatically set by default.) uri must be a URI object, params must be a hash, and header must be a hash.


Arguments:

  • (required) http: Net::HTTP object or compatible object
  • (required) path: String
  • (optional) params: Hash containing parameters
  • (optional) header: Hash containing header strings
Returns:(same as Net::HTTP::post_form)

[Source]

     # File lib/bio/command.rb, line 775
775:   def http_post_form(http, path, params = nil, header = {})
776:     data = make_cgi_params(params)
777: 
778:     hash = {
779:       'Content-Type'   => 'application/x-www-form-urlencoded',
780:       'Content-Length' => data.length.to_s
781:     }
782:     hash.update(header)
783: 
784:     http.post(path, data, hash)
785:   end

Builds parameter string for from Hash of parameters for application/x-www-form-urlencoded.


Arguments:

  • (required) params: Hash containing parameters
Returns:String

[Source]

     # File lib/bio/command.rb, line 829
829:   def make_cgi_params(params)
830:     data = ""
831:     case params
832:     when Hash
833:       data = params.map do |key, val|
834:         make_cgi_params_key_value(key, val)
835:       end.join('&')
836:     when Array
837:       case params.first
838:       when Hash
839:         data = params.map do |hash|
840:           hash.map do |key, val|
841:             make_cgi_params_key_value(key, val)
842:           end
843:         end.join('&')
844:       when Array
845:         data = params.map do |key, val|
846:           make_cgi_params_key_value(key, val)
847:         end.join('&')
848:       when String
849:         data = params.map do |str|
850:           key, val = str.split(/\=/, 2)
851:           if val then
852:             make_cgi_params_key_value(key, val)
853:           else
854:             CGI.escape(str)
855:           end
856:         end.join('&')
857:       end
858:     when String
859:       data = URI.escape(params.strip)
860:     end
861:     return data
862:   end

Builds parameter string for from a key string and a value (or values) for application/x-www-form-urlencoded.


Arguments:

Returns:String

[Source]

     # File lib/bio/command.rb, line 872
872:   def make_cgi_params_key_value(key, value)
873:     result = []
874:     case value
875:     when Array
876:       value.each do |val|
877:         result << [key, val].map {|x| CGI.escape(x.to_s) }.join('=')
878:       end
879:     else
880:       result << [key, value].map {|x| CGI.escape(x.to_s) }.join('=')
881:     end
882:     return result
883:   end

Generate command line string with special characters escaped.


Arguments:

  • (required) ary: Array containing String objects
Returns:String object

[Source]

     # File lib/bio/command.rb, line 134
134:   def make_command_line(ary)
135:     if windows_platform? then
136:       make_command_line_windows(ary)
137:     else
138:       make_command_line_unix(ary)
139:     end
140:   end

Generate command line string with special characters escaped for UNIX shells.


Arguments:

  • (required) ary: Array containing String objects
Returns:String object

[Source]

     # File lib/bio/command.rb, line 158
158:   def make_command_line_unix(ary)
159:     ary.collect { |str| escape_shell_unix(str) }.join(" ")
160:   end

Generate command line string with special characters escaped for cmd.exe on Windows.


Arguments:

  • (required) ary: Array containing String objects
Returns:String object

[Source]

     # File lib/bio/command.rb, line 148
148:   def make_command_line_windows(ary)
149:     ary.collect { |str| escape_shell_windows(str) }.join(" ")
150:   end

Backport of Dir.mktmpdir in Ruby 1.9.

Same as Dir.mktmpdir(prefix_suffix) in Ruby 1.9.


Arguments:

  • (optional) prefix_suffix: String (or Array, etc.)
  • (optional) tmpdir: String: temporary directory‘s path

[Source]

     # File lib/bio/command.rb, line 573
573:   def mktmpdir(prefix_suffix = nil, tmpdir = nil, &block)
574:     begin
575:       Dir.mktmpdir(prefix_suffix, tmpdir, &block)
576:     rescue NoMethodError
577:       # backported from Ruby 1.9.2-preview1.
578:       # ***** Below is excerpted from Ruby 1.9.2-preview1's lib/tmpdir.rb ****
579:       # ***** Be careful about copyright. ****
580:       case prefix_suffix
581:       when nil
582:         prefix = "d"
583:         suffix = ""
584:       when String
585:         prefix = prefix_suffix
586:         suffix = ""
587:       when Array
588:         prefix = prefix_suffix[0]
589:         suffix = prefix_suffix[1]
590:       else
591:         raise ArgumentError, "unexpected prefix_suffix: #{prefix_suffix.inspect}"
592:       end
593:       tmpdir ||= Dir.tmpdir
594:       t = Time.now.strftime("%Y%m%d")
595:       n = nil
596:       begin
597:         path = "#{tmpdir}/#{prefix}#{t}-#{$$}-#{rand(0x100000000).to_s(36)}"
598:         path << "-#{n}" if n
599:         path << suffix
600:         Dir.mkdir(path, 0700)
601:       rescue Errno::EEXIST
602:         n ||= 0
603:         n += 1
604:         retry
605:       end
606: 
607:       if block_given?
608:         begin
609:           yield path
610:         ensure
611:           remove_entry_secure path
612:         end
613:       else
614:         path
615:       end
616:       # ***** Above is excerpted from Ruby 1.9.2-preview1's lib/tmpdir.rb ****
617:     end
618:   end

Same as:

  Net::HTTP.new(address, port)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set.


Arguments:

  • (required) address: String containing host name or IP address
  • (optional) port: port (sanme as Net::HTTP::start)
Returns:(same as Net::HTTP.new except for proxy support)

[Source]

     # File lib/bio/command.rb, line 745
745:   def new_http(address, port = 80)
746:     uri = URI.parse("http://#{address}:#{port}")
747:     # Note: URI#find_proxy is an unofficial method defined in open-uri.rb.
748:     # If the spec of open-uri.rb would be changed, we should change below.
749:     if proxyuri = uri.find_proxy then
750:       raise 'Non-HTTP proxy' if proxyuri.class != URI::HTTP
751:       Net::HTTP.new(address, port, proxyuri.host, proxyuri.port)
752:     else
753:       Net::HTTP.new(address, port)
754:     end
755:   end

CAUTION Bio::Command INTERNAL USE ONLY. Users must NOT use the method. The method will be removed when it is not needed.

Checks if the OS does not support fork(2) system call. When not supported, it returns true. When supported or unknown, it returns false or nil.

Known issues:

  • It might make a mistake in minor platforms/architectures/interpreters.

Returns:true, false or nil.

[Source]

    # File lib/bio/command.rb, line 80
80:   def no_fork?
81:     if (defined?(@@no_fork) && @@no_fork) or
82:         windows_platform? or /java/i =~ RUBY_PLATFORM then
83:       true
84:     else
85:       false
86:     end
87:   end

Same as: Net::HTTP.post_form(uri, params) and it uses proxy if an environment variable (same as OpenURI.open_uri) is set. In addition, header can be set. (Note that Content-Type and Content-Length are automatically set by default.) uri must be a URI object, params must be a hash, and header must be a hash.


Arguments:

  • (required) uri: URI object or String
  • (optional) params: Hash containing parameters
  • (optional) header: Hash containing header strings
Returns:(same as Net::HTTP::post_form)

[Source]

     # File lib/bio/command.rb, line 804
804:   def post_form(uri, params = nil, header = {})
805:     unless uri.is_a?(URI)
806:       uri = URI.parse(uri)
807:     end
808: 
809:     data = make_cgi_params(params)
810: 
811:     hash = {
812:       'Content-Type'   => 'application/x-www-form-urlencoded',
813:       'Content-Length' => data.length.to_s
814:     }
815:     hash.update(header)
816: 
817:     start_http(uri.host, uri.port) do |http|
818:       http.post(uri.path, data, hash)
819:     end
820:   end

Executes the program with the query (String) given to the standard input, waits the program termination, and returns the output data printed to the standard output as a string.

Automatically select popen for Ruby 1.9 or Windows environment and fork for the others.

Available options:

  :chdir => "path" : changes working directory to the specified path.

Arguments:

  • (required) cmd: Array containing String objects
  • (optional) query: String
  • (optional) options: Hash
Returns:String or nil

[Source]

     # File lib/bio/command.rb, line 442
442:   def query_command(cmd, query = nil, options = {})
443:     if RUBY_VERSION >= "1.9.0" then
444:       return query_command_popen(cmd, query, options)
445:     elsif no_fork? then
446:       query_command_popen(cmd, query, options)
447:     else
448:       begin
449:         query_command_fork(cmd, query, options)
450:       rescue NotImplementedError
451:         # fork(2) not implemented
452:         @@no_fork = true
453:         query_command_fork(cmd, query, options)
454:       end
455:     end
456:   end

This method is internally called from the query_command method. In normal case, use query_command, and do not call this method directly.

Executes the program with the query (String) given to the standard input, waits the program termination, and returns the output data printed to the standard output as a string.

Fork (by using IO.popen("-")) and exec is used to execute the program.

See the document of query_command for available options.

See the document of call_command_fork for the security and Ruby version specific issues.


Arguments:

  • (required) cmd: Array containing String objects
  • (optional) query: String
  • (optional) options: Hash
Returns:String or nil

[Source]

     # File lib/bio/command.rb, line 507
507:   def query_command_fork(cmd, query = nil, options = {})
508:     ret = nil
509:     call_command_fork(cmd, options) do |io|
510:       io.sync = true
511:       io.print query if query
512:       io.close_write
513:       ret = io.read
514:     end
515:     ret
516:   end

Executes the program via Open3.popen3 with the query (String) given to the stain, waits the program termination, and returns the data from stdout and stderr as an array of the strings.

You would use this method only when you really need to get stderr.


Arguments:

  • (required) cmd: Array containing String objects
  • (optional) query: String
Returns:Array containing 2 objects: output string (or nil) and stderr string (or nil)

[Source]

     # File lib/bio/command.rb, line 529
529:   def query_command_open3(cmd, query = nil)
530:     errorlog = nil
531:     cmd = safe_command_line_array(cmd)
532:     Open3.popen3(*cmd) do |pin, pout, perr|
533:       perr.sync = true
534:       t = Thread.start { errorlog = perr.read }
535:       begin
536:         pin.print query if query
537:         pin.close
538:         output = pout.read
539:       ensure
540:         t.join
541:       end
542:       [ output, errorlog ]
543:     end
544:   end

This method is internally called from the query_command method. In normal case, use query_command, and do not call this method directly.

Executes the program with the query (String) given to the standard input, waits the program termination, and returns the output data printed to the standard output as a string.

See the document of query_command for available options.

See the document of call_command_popen for the security and Ruby version specific issues.


Arguments:

  • (required) cmd: Array containing String objects
  • (optional) query: String
  • (optional) options: Hash
Returns:String or nil

[Source]

     # File lib/bio/command.rb, line 476
476:   def query_command_popen(cmd, query = nil, options = {})
477:     ret = nil
478:     call_command_popen(cmd, options) do |io|
479:       io.sync = true
480:       io.print query if query
481:       io.close_write
482:       ret = io.read
483:     end
484:     ret
485:   end

Same as OpenURI.open_uri(uri).read and it uses proxy if an environment variable (same as OpenURI.open_uri) is set.


Arguments:

  • (required) uri: URI object or String
Returns:String

[Source]

     # File lib/bio/command.rb, line 706
706:   def read_uri(uri)
707:     OpenURI.open_uri(uri).read
708:   end

Same as FileUtils.remove_entry_secure after Ruby 1.8.3. In Ruby 1.8.2 or previous version, it only shows warning message and does nothing.

It is strongly recommended using Ruby 1.8.5 or later.


Arguments:

  • (required) path: String
  • (optional) force: boolean

[Source]

     # File lib/bio/command.rb, line 555
555:   def remove_entry_secure(path, force = false)
556:     begin
557:       FileUtils.remove_entry_secure(path, force)
558:     rescue NoMethodError
559:       warn "The temporary file or directory is not removed because of the lack of FileUtils.remove_entry_secure. Use Ruby 1.8.3 or later (1.8.5 or later is strongly recommended): #{path}"
560:       nil
561:     end
562:   end

Returns an Array of command-line command and arguments that can be safely passed to Kernel.exec etc. If the given array is already safe (or empty), returns the given array.


Arguments:

  • (required) ary: Array
Returns:Array

[Source]

     # File lib/bio/command.rb, line 169
169:   def safe_command_line_array(ary)
170:     ary = ary.to_ary
171:     return ary if ary.size >= 2 or ary.empty?
172:     if ary.size != 1 then
173:       raise 'Bug: assersion of ary.size == 1 failed'
174:     end
175:     arg0 = ary[0]
176:     begin
177:       arg0 = arg0.to_ary
178:     rescue NoMethodError
179:       arg0 = [ arg0, arg0 ]
180:     end
181:     [ arg0 ]
182:   end

Same as:

  Net::HTTP.start(address, port)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set.


Arguments:

  • (required) address: String containing host name or IP address
  • (optional) port: port (sanme as Net::HTTP::start)
Returns:(same as Net::HTTP::start except for proxy support)

[Source]

     # File lib/bio/command.rb, line 721
721:   def start_http(address, port = 80, &block)
722:     uri = URI.parse("http://#{address}:#{port}")
723:     # Note: URI#find_proxy is an unofficial method defined in open-uri.rb.
724:     # If the spec of open-uri.rb would be changed, we should change below.
725:     if proxyuri = uri.find_proxy then
726:       raise 'Non-HTTP proxy' if proxyuri.class != URI::HTTP
727:       http = Net::HTTP.Proxy(proxyuri.host, proxyuri.port)
728:     else
729:       http = Net::HTTP
730:     end
731:     http.start(address, port, &block)
732:   end

CAUTION Bio::Command INTERNAL USE ONLY. Users must NOT use the method. The method will be removed when it is not needed.

Checks if the program is running on Microsoft Windows. If Windows, returns true. Otherwise, returns false. Note that Cygwin is not treated as Windows.

Known issues:

  • It might make a mistake in minor platforms/architectures/interpreters.
  • When running JRuby on Cygwin, the result is unknown.

Returns:true or false

[Source]

    # File lib/bio/command.rb, line 50
50:   def windows_platform?
51:     case RUBY_PLATFORM
52:     when /(?:mswin|bccwin|mingw)(?:32|64)/i
53:       true
54:     when /java/i
55:       # Reference: Redmine's platform.rb
56:       # http://www.redmine.org/projects/redmine/repository/revisions/1753/entry/trunk/lib/redmine/platform.rb
57:       if /windows/i =~ (ENV['OS'] || ENV['os']).to_s then
58:         true
59:       else
60:         false
61:       end
62:     else
63:       false
64:     end
65:   end

[Validate]