From 1d3817e0074901b7ed5b93248e7d4c7feaf63a03 Mon Sep 17 00:00:00 2001 From: Austin Ziegler Date: Sat, 27 Mar 2010 01:57:44 -0400 Subject: [PATCH] Reformat and documentation update of Net::BER and Net::BER::BERParser. --- History.txt | 3 +- lib/net/ber.rb | 392 ++++++++++++++++++++++++++++++-------- lib/net/ber/ber_parser.rb | 252 ++++++++++++++---------- 3 files changed, 472 insertions(+), 175 deletions(-) diff --git a/History.txt b/History.txt index bd53795..b066a73 100644 --- a/History.txt +++ b/History.txt @@ -25,7 +25,8 @@ * Extended unit testing: * Added some unit tests for the BER core extensions. * Code clean-up: - * Made the formatting of Net::LDAP code consistent across all files. + * Made the formatting of code consistent across all files. + * Removed Net::BER::BERParser::TagClasses as it does not appear to be used. * Replaced calls to #to_a with calls to Kernel#Array; since Ruby 1.8.3, the default #to_a implementation has been deprecated and should be replaced either with calls to Kernel#Array or [value].flatten(1). diff --git a/lib/net/ber.rb b/lib/net/ber.rb index 26d01b4..cc4ec74 100644 --- a/lib/net/ber.rb +++ b/lib/net/ber.rb @@ -1,99 +1,339 @@ # NET::BER -# Mixes ASN.1/BER convenience methods into several standard classes. -# Also provides BER parsing functionality. -# -#---------------------------------------------------------------------------- +# Mixes ASN.1/BER convenience methods into several standard classes. Also +# provides BER parsing functionality. # +#-- # Copyright (C) 2006 by Francis Cianfrocca. All Rights Reserved. # # Gmail: garbagecat10 # -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. +# This program is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by the Free +# Software Foundation; either version 2 of the License, or (at your option) +# any later version. # -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY +# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +# for more details. # -# You should have received a copy of the GNU General Public License -# along with this program; if not, write to the Free Software -# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -# -#--------------------------------------------------------------------------- +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., 51 +# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#++ module Net + ## + # == Basic Encoding Rules (BER) Support Module + # + # Much of the text below is cribbed from Wikipedia: + # http://en.wikipedia.org/wiki/Basic_Encoding_Rules + # + # The ITU Specification is also worthwhile reading: + # http://www.itu.int/ITU-T/studygroups/com17/languages/X.690-0207.pdf + # + # The Basic Encoding Rules were the original rules laid out by the ASN.1 + # standard for encoding abstract information into a concrete data stream. + # The rules, collectively referred to as a transfer syntax in ASN.1 + # parlance, specify the exact octet sequences which are used to encode a + # given data item. The syntax defines such elements as: the + # representations for basic data types, the structure of length + # information, and the means for defining complex or compound types based + # on more primitive types. The BER syntax, along with two subsets of BER + # (the Canonical Encoding Rules and the Distinguished Encoding Rules), are + # defined by the ITU-T's X.690 standards document, which is part of the + # ASN.1 document series. + # + # == Encoding + # The BER format specifies a self-describing and self-delimiting format + # for encoding ASN.1 data structures. Each data element is encoded as a + # type identifier, a length description, the actual data elements, and + # where necessary, an end-of-content marker. This format allows a receiver + # to decode the ASN.1 information from an incomplete stream, without + # requiring any pre-knowledge of the size, content, or semantic meaning of + # the data. + # + # + # + # == Protocol Data Units (PDU) + # Protocols are defined with schema represented in BER, such that a PDU + # consists of cascaded type-length-value encodings. + # + # === Type Tags + # BER type tags are represented as single octets (bytes). The lower five + # bits of the octet are tag identifier numbers and the upper three bits of + # the octet are used to distinguish the type as native to ASN.1, + # application-specific, context-specific, or private. See + # Net::BER::TAG_CLASS and Net::BER::ENCODING_TYPE for more information. + # + # If Class is set to Universal (0b00______), the value is of a type native + # to ASN.1 (e.g. INTEGER). The Application class (0b01______) is only + # valid for one specific application. Context_specific (0b10______) + # depends on the context and private (0b11_______) can be defined in + # private specifications + # + # If the primitive/constructed bit is zero (0b__0_____), it specifies that + # the value is primitive like an INTEGER. If it is one (0b__1_____), the + # value is a constructed value that contains type-length-value encoded + # types like a SET or a SEQUENCE. + # + # === Defined Universal (ASN.1 Native) Types + # There are a number of pre-defined universal (native) types. + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + #
NamePrimitive
Constructed		Number
EOC (End-of-Content)P0: 0 (0x0, 0b00000000)
BOOLEANP1: 1 (0x01, 0b00000001)
INTEGERP2: 2 (0x02, 0b00000010)
BIT STRINGP3: 3 (0x03, 0b00000011)
BIT STRINGC3: 35 (0x23, 0b00100011)
OCTET STRINGP4: 4 (0x04, 0b00000100)
OCTET STRINGC4: 36 (0x24, 0b00100100)
NULLP5: 5 (0x05, 0b00000101)
OBJECT IDENTIFIERP6: 6 (0x06, 0b00000110)
Object DescriptorP7: 7 (0x07, 0b00000111)
EXTERNALC8: 40 (0x28, 0b00101000)
REAL (float)P9: 9 (0x09, 0b00001001)
ENUMERATEDP10: 10 (0x0a, 0b00001010)
EMBEDDED PDVC11: 43 (0x2b, 0b00101011)
UTF8StringP12: 12 (0x0c, 0b00001100)
UTF8StringC12: 44 (0x2c, 0b00101100)
RELATIVE-OIDP13: 13 (0x0d, 0b00001101)
SEQUENCE and SEQUENCE OFC16: 48 (0x30, 0b00110000)
SET and SET OFC17: 49 (0x31, 0b00110001)
NumericStringP18: 18 (0x12, 0b00010010)
NumericStringC18: 50 (0x32, 0b00110010)
PrintableStringP19: 19 (0x13, 0b00010011)
PrintableStringC19: 51 (0x33, 0b00110011)
T61StringP20: 20 (0x14, 0b00010100)
T61StringC20: 52 (0x34, 0b00110100)
VideotexStringP21: 21 (0x15, 0b00010101)
VideotexStringC21: 53 (0x35, 0b00110101)
IA5StringP22: 22 (0x16, 0b00010110)
IA5StringC22: 54 (0x36, 0b00110110)
UTCTimeP23: 23 (0x17, 0b00010111)
UTCTimeC23: 55 (0x37, 0b00110111)
GeneralizedTimeP24: 24 (0x18, 0b00011000)
GeneralizedTimeC24: 56 (0x38, 0b00111000)
GraphicStringP25: 25 (0x19, 0b00011001)
GraphicStringC25: 57 (0x39, 0b00111001)
VisibleStringP26: 26 (0x1a, 0b00011010)
VisibleStringC26: 58 (0x3a, 0b00111010)
GeneralStringP27: 27 (0x1b, 0b00011011)
GeneralStringC27: 59 (0x3b, 0b00111011)
UniversalStringP28: 28 (0x1c, 0b00011100)
UniversalStringC28: 60 (0x3c, 0b00111100)
CHARACTER STRINGP29: 29 (0x1d, 0b00011101)
CHARACTER STRINGC29: 61 (0x3d, 0b00111101)
BMPStringP30: 30 (0x1e, 0b00011110)
BMPStringC30: 62 (0x3e, 0b00111110)
module BER VERSION = '0.1.0' - - #-- - # This condenses our nicely self-documenting ASN hashes down - # to an array for fast lookups. - # Scoped to be called as a module method, but not intended for - # user code to call. - # - def self.compile_syntax(syn) - out = [nil] * 256 - syn.each do |tclass, tclasses| - tagclass = {:universal=>0, :application=>64, :context_specific=>128, :private=>192} [tclass] - tclasses.each do |codingtype,codings| - encoding = {:primitive=>0, :constructed=>32} [codingtype] - codings.each {|tag, objtype| out[tagclass + encoding + tag] = objtype } - end - end - out - end - def to_ber - # Provisional implementation. - # We ASSUME that our incoming value is an array, and we - # use the Array#to_ber_oid method defined below. - # We probably should obsolete that method, actually, in - # and move the code here. - # WE ARE NOT CURRENTLY ENCODING THE BER-IDENTIFIER. - # This implementation currently hardcodes 6, the universal OID tag. - ary = @value.dup - first = ary.shift - raise Net::BER::BerError.new(" invalid OID" ) unless [0,1,2].include?(first) - first = first * 40 + ary.shift - ary.unshift first - oid = ary.pack("w*") - [6, oid.length].pack("CC") + oid - end - end -end - -module Net - module BER + ## # Used for BER-encoding the length and content bytes of a Fixnum integer # values. MAX_FIXNUM_SIZE = 0.size - class BerError < StandardError; end - - class BerIdentifiedString < String - attr_accessor :ber_identifier - def initialize args - super args - end - end - - class BerIdentifiedArray < Array - attr_accessor :ber_identifier - def initialize(*args) - super - end - end - - class BerIdentifiedNull - attr_accessor :ber_identifier - def to_ber - "\005\000" + ## + # BER tag classes are kept in bits seven and eight of the tag type + # octet. + # + # + # + # + # + # + # + #
BitmaskDefinition
0b00______Universal (ASN.1 Native) Types
0b01______Application Types
0b10______Context-Specific Types
0b11______Private Types
+ TAG_CLASS = { + :universal => 0b00000000, # 0 + :application => 0b01000000, # 64 + :context_specific => 0b10000000, # 128 + :private => 0b11000000, # 192 + } + + ## + # BER encoding type is kept in bit 6 of the tag type octet. + # + # + # + # + # + #
BitmaskDefinition
0b__0_____Primitive
0b__1_____Constructed
+ ENCODING_TYPE = { + :primitive => 0b00000000, # 0 + :constructed => 0b00100000, # 32 + } + + ## + # Accepts a hash of hashes describing a BER syntax and converts it into + # a byte-keyed object for fast BER conversion lookup. The resulting + # "compiled" syntax is used by Net::BER::BERParser. + # + # This method should be called only by client classes of Net::BER (e.g., + # Net::LDAP and Net::SNMP) and not by clients of those classes. + # + # The hash-based syntax uses TAG_CLASS keys that contain hashes of + # ENCODING_TYPE keys that contain tag numbers with object type markers. + # + # : => { + # : => { + # => + # }, + # }, + # + # === Permitted Object Types + # :string:: A string value, represented as BerIdentifiedString. + # :integer:: An integer value, represented with Fixnum. + # :oid:: An Object Identifier value; see X.690 section + # 8.19. Currently represented with a standard array, + # but may be better represented as a + # BerIdentifiedOID object. + # :array:: A sequence, represented as BerIdentifiedArray. + # :boolean:: A boolean value, represented as +true+ or +false+. + # :null:: A null value, represented as BerIdentifiedNull. + # + # === Example + # Net::LDAP defines its ASN.1 BER syntax something like this: + # + # class Net::LDAP + # AsnSyntax = Net::BER.compile_syntax({ + # :application => { + # :primitive => { + # 2 => :null, + # }, + # :constructed => { + # 0 => :array, + # # ... + # }, + # }, + # :context_specific => { + # :primitive => { + # 0 => :string, + # # ... + # }, + # :constructed => { + # 0 => :array, + # # ... + # }, + # } + # }) + # end + # + # NOTE:: For readability and formatting purposes, Net::LDAP and its + # siblings actually construct their syntaxes more deliberately, + # as shown below. Since a hash is passed in the end in any case, + # the format does not matter. + # + # primitive = { 2 => :null } + # constructed = { + # 0 => :array, + # # ... + # } + # application = { + # :primitive => primitive, + # :constructed => constructed + # } + # + # primitive = { + # 0 => :string, + # # ... + # } + # constructed = { + # 0 => :array, + # # ... + # } + # context_specific = { + # :primitive => primitive, + # :constructed => constructed + # } + # AsnSyntax = Net::BER.compile_syntax(:application => application, + # :context_specific => context_specific) + def self.compile_syntax(syntax) + # TODO 20100327 AZ: Should we be allocating an array of 256 values + # that will either be +nil+ or an object type symbol, or should we + # allocate an empty Hash since unknown values return +nil+ anyway? + out = [ nil ] * 256 + syntax.each do |tag_class_id, encodings| + tag_class = TAG_CLASS[tag_class_id] + encodings.each do |encoding_id, classes| + encoding = ENCODING_TYPE[encoding_id] + object_class = tag_class + encoding + classes.each do |number, object_type| + out[object_class + number] = object_type + end + end end + out end end end +class Net::BER::BerError < RuntimeError; end + +## +# An Array object with a BER identifier attached. +class Net::BER::BerIdentifiedArray < Array + attr_accessor :ber_identifier + + def initialize(*args) + super + end +end + +## +# A BER object identifier. +class Net::BER::BerIdentifiedOid + attr_accessor :ber_identifier + + def initialize(oid) + if oid.is_a?(String) + oid = oid.split(/\./).map {|s| s.to_i } + end + @value = oid + end + + def to_ber + to_ber_oid + end + + def to_ber_oid + @value.to_ber_oid + end + + def to_s + @value.join(".") + end + + def to_arr + @value.dup + end +end + +## +# A String object with a BER identifier attached. +class Net::BER::BerIdentifiedString < String + attr_accessor :ber_identifier + def initialize args + super args + end +end + +module Net::BER + ## + # A BER null object. + class BerIdentifiedNull + attr_accessor :ber_identifier + def to_ber + "\005\000" + end + end + + ## + # The default BerIdentifiedNull object. + Null = Net::BER::BerIdentifiedNull.new +end + require 'net/ber/core_ext' diff --git a/lib/net/ber/ber_parser.rb b/lib/net/ber/ber_parser.rb index f24e550..fec2f67 100644 --- a/lib/net/ber/ber_parser.rb +++ b/lib/net/ber/ber_parser.rb @@ -1,112 +1,168 @@ require 'stringio' -module Net - module BER - module BERParser - VERSION = '0.1.0' +## +# Implements Basic Encoding Rules parsing to be mixed into types as needed. +module Net::BER::BERParser + primitive = { + 1 => :boolean, + 2 => :integer, + 4 => :string, + 5 => :null, + 6 => :oid, + 10 => :integer, + 13 => :string # (relative OID) + } + constructed = { + 16 => :array, + 17 => :array + } + universal = { :primitive => primitive, :constructed => constructed } - # The order of these follows the class-codes in BER. - # Maybe this should have been a hash. - TagClasses = [:universal, :application, :context_specific, :private] + primitive = { 10 => :integer } + context = { :primitive => primitive } - BuiltinSyntax = Net::BER.compile_syntax( { - :universal => { - :primitive => { - 1 => :boolean, - 2 => :integer, - 4 => :string, - 5 => :null, - 6 => :oid, - 10 => :integer, - 13 => :string # (relative OID) - }, - :constructed => { - 16 => :array, - 17 => :array - } - }, - :context_specific => { - :primitive => { - 10 => :integer - } - } - }) + # The universal, built-in ASN.1 BER syntax. + BuiltinSyntax = Net::BER.compile_syntax(:universal => universal, + :context_specific => context) - def read_ber syntax=nil - # TODO: clean this up so it works properly with partial - # packets coming from streams that don't block when - # we ask for more data (like StringIOs). At it is, - # this can throw TypeErrors and other nasties. - - id = getbyte or return nil # don't trash this value, we'll use it later + ## + # This is an extract of our BER object parsing to simplify our + # understanding of how we parse basic BER object types. + def parse_ber_object(syntax, id, data) + # Find the object type from either the provided syntax lookup table or + # the built-in syntax lookup table. + # + # This exceptionally clever bit of code is verrrry slow. + object_type = (syntax && syntax[id]) || BuiltinSyntax[id] - n = getbyte - lengthlength,contentlength = if n <= 127 - [1,n] - else - # Replaced the inject because it profiles hot. - # j = (0...(n & 127)).inject(0) {|mem,x| mem = (mem << 8) + getc} - j = 0 - read( n & 127 ).each_byte {|n1| j = (j << 8) + n1} - [1 + (n & 127), j] - end - - newobj = read contentlength - - # This exceptionally clever and clear bit of code is verrrry slow. - objtype = (syntax && syntax[id]) || BuiltinSyntax[id] - - # == is expensive so sort this if/else so the common cases are at the top. - obj = if objtype == :string - #(newobj || "").dup - s = BerIdentifiedString.new( newobj || "" ) - s.ber_identifier = id - s - elsif objtype == :integer - j = 0 - newobj.each_byte {|b| j = (j << 8) + b} - j - elsif objtype == :oid - # cf X.690 pgh 8.19 for an explanation of this algorithm. - # Potentially not good enough. We may need a BerIdentifiedOid - # as a subclass of BerIdentifiedArray, to get the ber identifier - # and also a to_s method that produces the familiar dotted notation. - oid = newobj.unpack("w*") - f = oid.shift - g = if f < 40 + # == is expensive so sort this so the common cases are at the top. + if object_type == :string + s = Net::BER::BerIdentifiedString.new(data || "") + s.ber_identifier = id + s + elsif object_type == :integer + j = 0 + data.each_byte { |b| j = (j << 8) + b } + j + elsif object_type == :oid + # See X.690 pgh 8.19 for an explanation of this algorithm. + # This is potentially not good enough. We may need a + # BerIdentifiedOid as a subclass of BerIdentifiedArray, to + # get the ber identifier and also a to_s method that produces + # the familiar dotted notation. + oid = data.unpack("w*") + f = oid.shift + g = if f < 40 [0, f] elsif f < 80 - [1, f-40] + [1, f - 40] else - [2, f-80] # f-80 can easily be > 80. What a weird optimization. + # f - 80 can easily be > 80. What a weird optimization. + [2, f - 80] end - oid.unshift g.last - oid.unshift g.first - oid - elsif objtype == :array - #seq = [] - seq = BerIdentifiedArray.new - seq.ber_identifier = id - sio = StringIO.new( newobj || "" ) - # Interpret the subobject, but note how the loop - # is built: nil ends the loop, but false (a valid - # BER value) does not! - while (e = sio.read_ber(syntax)) != nil - seq << e - end - seq - elsif objtype == :boolean - newobj != "\000" - elsif objtype == :null - n = BerIdentifiedNull.new - n.ber_identifier = id - n - else - raise BerError, "unsupported object type: id=0x#{id.to_s(16)}" - end - - obj + oid.unshift g.last + oid.unshift g.first + # Net::BER::BerIdentifiedOid.new(oid) + oid + elsif object_type == :array + seq = Net::BER::BerIdentifiedArray.new + seq.ber_identifier = id + sio = StringIO.new(data || "") + # Interpret the subobject, but note how the loop is built: + # nil ends the loop, but false (a valid BER value) does not! + while (e = sio.read_ber(syntax)) != nil + seq << e end + seq + elsif object_type == :boolean + data != "\000" + elsif object_type == :null + n = Net::BER::BerIdentifiedNull.new + n.ber_identifier = id + n + else + raise Net::BER::BerError, "Unsupported object type: id=#{id}" end end + private :parse_ber_object + + ## + # This is an extract of how our BER object length parsing is done to + # simplify the primary call. This is defined in X.690 section 8.1.3. + # + # The BER length will either be a single byte or up to 126 bytes in + # length. There is a special case of a BER length indicating that the + # content-length is undefined and will be identified by the presence of + # two null values (0x00 0x00). + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + # + #
RangeLength
0x00 -- 0x7f
0b00000000 -- 0b01111111		0 - 127 bytes
0x80
0b10000000		Indeterminate (end-of-content marker required)
0x81 -- 0xfe
0b10000001 -- 0b11111110		1 - 126 bytes of length as an integer value
0xff
0b11111111		Illegal (reserved for future expansion)
+ # + #-- + # This has been modified from the version that was previously inside + # #read_ber to handle both the indeterminate terminator case and the + # invalid BER length case. Because the "lengthlength" value was not used + # inside of #read_ber, we no longer return it. + def read_ber_length + n = getbyte + + if n <= 0x7f + n + elsif n == 0x80 + -1 + elsif n == 0xff + raise Net::BER::BerError, "Invalid BER length 0xFF detected." + else + v = 0 + read(n & 0x7f).each_byte do |b| + v = (v << 8) + b + end + + v + end + end + private :read_ber_length + + ## + # Reads a BER object from the including object. Requires that #getbyte is + # implemented on the including object and that it returns a Fixnum value. + # Also requires #read(bytes) to work. + # + # This does not work with non-blocking I/O. + def read_ber(syntax = nil) + # TODO: clean this up so it works properly with partial packets coming + # from streams that don't block when we ask for more data (like + # StringIOs). At it is, this can throw TypeErrors and other nasties. + + id = getbyte or return nil # don't trash this value, we'll use it later + content_length = read_ber_length + + if -1 == content_length + raise Net::BER::BerError, "Indeterminite BER content length not implemented." + else + data = read(content_length) + end + + parse_ber_object(syntax, id, data) + end end