Reformat and documentation update of Net::BER and Net::BER::BERParser.

master
Austin Ziegler 2010-03-27 01:57:44 -04:00
parent 28d58cd581
commit 1d3817e007
3 changed files with 472 additions and 175 deletions

View File

@ -25,7 +25,8 @@
* Extended unit testing:
* Added some unit tests for the BER core extensions.
* Code clean-up:
* Made the formatting of Net::LDAP code consistent across all files.
* Made the formatting of code consistent across all files.
* Removed Net::BER::BERParser::TagClasses as it does not appear to be used.
* Replaced calls to #to_a with calls to Kernel#Array; since Ruby 1.8.3, the
default #to_a implementation has been deprecated and should be replaced
either with calls to Kernel#Array or [value].flatten(1).

View File

@ -1,99 +1,339 @@
# NET::BER
# Mixes ASN.1/BER convenience methods into several standard classes.
# Also provides BER parsing functionality.
#
#----------------------------------------------------------------------------
# Mixes ASN.1/BER convenience methods into several standard classes. Also
# provides BER parsing functionality.
#
#--
# Copyright (C) 2006 by Francis Cianfrocca. All Rights Reserved.
#
# Gmail: garbagecat10
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the Free
# Software Foundation; either version 2 of the License, or (at your option)
# any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#
#---------------------------------------------------------------------------
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc., 51
# Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#++
module Net
##
# == Basic Encoding Rules (BER) Support Module
#
# Much of the text below is cribbed from Wikipedia:
# http://en.wikipedia.org/wiki/Basic_Encoding_Rules
#
# The ITU Specification is also worthwhile reading:
# http://www.itu.int/ITU-T/studygroups/com17/languages/X.690-0207.pdf
#
# The Basic Encoding Rules were the original rules laid out by the ASN.1
# standard for encoding abstract information into a concrete data stream.
# The rules, collectively referred to as a transfer syntax in ASN.1
# parlance, specify the exact octet sequences which are used to encode a
# given data item. The syntax defines such elements as: the
# representations for basic data types, the structure of length
# information, and the means for defining complex or compound types based
# on more primitive types. The BER syntax, along with two subsets of BER
# (the Canonical Encoding Rules and the Distinguished Encoding Rules), are
# defined by the ITU-T's X.690 standards document, which is part of the
# ASN.1 document series.
#
# == Encoding
# The BER format specifies a self-describing and self-delimiting format
# for encoding ASN.1 data structures. Each data element is encoded as a
# type identifier, a length description, the actual data elements, and
# where necessary, an end-of-content marker. This format allows a receiver
# to decode the ASN.1 information from an incomplete stream, without
# requiring any pre-knowledge of the size, content, or semantic meaning of
# the data.
#
# <Type | Length | Value [| End-of-Content]>
#
# == Protocol Data Units (PDU)
# Protocols are defined with schema represented in BER, such that a PDU
# consists of cascaded type-length-value encodings.
#
# === Type Tags
# BER type tags are represented as single octets (bytes). The lower five
# bits of the octet are tag identifier numbers and the upper three bits of
# the octet are used to distinguish the type as native to ASN.1,
# application-specific, context-specific, or private. See
# Net::BER::TAG_CLASS and Net::BER::ENCODING_TYPE for more information.
#
# If Class is set to Universal (0b00______), the value is of a type native
# to ASN.1 (e.g. INTEGER). The Application class (0b01______) is only
# valid for one specific application. Context_specific (0b10______)
# depends on the context and private (0b11_______) can be defined in
# private specifications
#
# If the primitive/constructed bit is zero (0b__0_____), it specifies that
# the value is primitive like an INTEGER. If it is one (0b__1_____), the
# value is a constructed value that contains type-length-value encoded
# types like a SET or a SEQUENCE.
#
# === Defined Universal (ASN.1 Native) Types
# There are a number of pre-defined universal (native) types.
#
# <table>
# <tr><th>Name</th><th>Primitive<br />Constructed</th><th>Number</th></tr>
# <tr><th>EOC (End-of-Content)</th><th>P</th><td>0: 0 (0x0, 0b00000000)</td></tr>
# <tr><th>BOOLEAN</th><th>P</th><td>1: 1 (0x01, 0b00000001)</td></tr>
# <tr><th>INTEGER</th><th>P</th><td>2: 2 (0x02, 0b00000010)</td></tr>
# <tr><th>BIT STRING</th><th>P</th><td>3: 3 (0x03, 0b00000011)</td></tr>
# <tr><th>BIT STRING</th><th>C</th><td>3: 35 (0x23, 0b00100011)</td></tr>
# <tr><th>OCTET STRING</th><th>P</th><td>4: 4 (0x04, 0b00000100)</td></tr>
# <tr><th>OCTET STRING</th><th>C</th><td>4: 36 (0x24, 0b00100100)</td></tr>
# <tr><th>NULL</th><th>P</th><td>5: 5 (0x05, 0b00000101)</td></tr>
# <tr><th>OBJECT IDENTIFIER</th><th>P</th><td>6: 6 (0x06, 0b00000110)</td></tr>
# <tr><th>Object Descriptor</th><th>P</th><td>7: 7 (0x07, 0b00000111)</td></tr>
# <tr><th>EXTERNAL</th><th>C</th><td>8: 40 (0x28, 0b00101000)</td></tr>
# <tr><th>REAL (float)</th><th>P</th><td>9: 9 (0x09, 0b00001001)</td></tr>
# <tr><th>ENUMERATED</th><th>P</th><td>10: 10 (0x0a, 0b00001010)</td></tr>
# <tr><th>EMBEDDED PDV</th><th>C</th><td>11: 43 (0x2b, 0b00101011)</td></tr>
# <tr><th>UTF8String</th><th>P</th><td>12: 12 (0x0c, 0b00001100)</td></tr>
# <tr><th>UTF8String</th><th>C</th><td>12: 44 (0x2c, 0b00101100)</td></tr>
# <tr><th>RELATIVE-OID</th><th>P</th><td>13: 13 (0x0d, 0b00001101)</td></tr>
# <tr><th>SEQUENCE and SEQUENCE OF</th><th>C</th><td>16: 48 (0x30, 0b00110000)</td></tr>
# <tr><th>SET and SET OF</th><th>C</th><td>17: 49 (0x31, 0b00110001)</td></tr>
# <tr><th>NumericString</th><th>P</th><td>18: 18 (0x12, 0b00010010)</td></tr>
# <tr><th>NumericString</th><th>C</th><td>18: 50 (0x32, 0b00110010)</td></tr>
# <tr><th>PrintableString</th><th>P</th><td>19: 19 (0x13, 0b00010011)</td></tr>
# <tr><th>PrintableString</th><th>C</th><td>19: 51 (0x33, 0b00110011)</td></tr>
# <tr><th>T61String</th><th>P</th><td>20: 20 (0x14, 0b00010100)</td></tr>
# <tr><th>T61String</th><th>C</th><td>20: 52 (0x34, 0b00110100)</td></tr>
# <tr><th>VideotexString</th><th>P</th><td>21: 21 (0x15, 0b00010101)</td></tr>
# <tr><th>VideotexString</th><th>C</th><td>21: 53 (0x35, 0b00110101)</td></tr>
# <tr><th>IA5String</th><th>P</th><td>22: 22 (0x16, 0b00010110)</td></tr>
# <tr><th>IA5String</th><th>C</th><td>22: 54 (0x36, 0b00110110)</td></tr>
# <tr><th>UTCTime</th><th>P</th><td>23: 23 (0x17, 0b00010111)</td></tr>
# <tr><th>UTCTime</th><th>C</th><td>23: 55 (0x37, 0b00110111)</td></tr>
# <tr><th>GeneralizedTime</th><th>P</th><td>24: 24 (0x18, 0b00011000)</td></tr>
# <tr><th>GeneralizedTime</th><th>C</th><td>24: 56 (0x38, 0b00111000)</td></tr>
# <tr><th>GraphicString</th><th>P</th><td>25: 25 (0x19, 0b00011001)</td></tr>
# <tr><th>GraphicString</th><th>C</th><td>25: 57 (0x39, 0b00111001)</td></tr>
# <tr><th>VisibleString</th><th>P</th><td>26: 26 (0x1a, 0b00011010)</td></tr>
# <tr><th>VisibleString</th><th>C</th><td>26: 58 (0x3a, 0b00111010)</td></tr>
# <tr><th>GeneralString</th><th>P</th><td>27: 27 (0x1b, 0b00011011)</td></tr>
# <tr><th>GeneralString</th><th>C</th><td>27: 59 (0x3b, 0b00111011)</td></tr>
# <tr><th>UniversalString</th><th>P</th><td>28: 28 (0x1c, 0b00011100)</td></tr>
# <tr><th>UniversalString</th><th>C</th><td>28: 60 (0x3c, 0b00111100)</td></tr>
# <tr><th>CHARACTER STRING</th><th>P</th><td>29: 29 (0x1d, 0b00011101)</td></tr>
# <tr><th>CHARACTER STRING</th><th>C</th><td>29: 61 (0x3d, 0b00111101)</td></tr>
# <tr><th>BMPString</th><th>P</th><td>30: 30 (0x1e, 0b00011110)</td></tr>
# <tr><th>BMPString</th><th>C</th><td>30: 62 (0x3e, 0b00111110)</td></tr>
# </table>
module BER
VERSION = '0.1.0'
#--
# This condenses our nicely self-documenting ASN hashes down
# to an array for fast lookups.
# Scoped to be called as a module method, but not intended for
# user code to call.
#
def self.compile_syntax(syn)
out = [nil] * 256
syn.each do |tclass, tclasses|
tagclass = {:universal=>0, :application=>64, :context_specific=>128, :private=>192} [tclass]
tclasses.each do |codingtype,codings|
encoding = {:primitive=>0, :constructed=>32} [codingtype]
codings.each {|tag, objtype| out[tagclass + encoding + tag] = objtype }
end
end
out
end
def to_ber
# Provisional implementation.
# We ASSUME that our incoming value is an array, and we
# use the Array#to_ber_oid method defined below.
# We probably should obsolete that method, actually, in
# and move the code here.
# WE ARE NOT CURRENTLY ENCODING THE BER-IDENTIFIER.
# This implementation currently hardcodes 6, the universal OID tag.
ary = @value.dup
first = ary.shift
raise Net::BER::BerError.new(" invalid OID" ) unless [0,1,2].include?(first)
first = first * 40 + ary.shift
ary.unshift first
oid = ary.pack("w*")
[6, oid.length].pack("CC") + oid
end
end
end
module Net
module BER
##
# Used for BER-encoding the length and content bytes of a Fixnum integer
# values.
MAX_FIXNUM_SIZE = 0.size
class BerError < StandardError; end
class BerIdentifiedString < String
attr_accessor :ber_identifier
def initialize args
super args
end
end
class BerIdentifiedArray < Array
attr_accessor :ber_identifier
def initialize(*args)
super
end
end
class BerIdentifiedNull
attr_accessor :ber_identifier
def to_ber
"\005\000"
##
# BER tag classes are kept in bits seven and eight of the tag type
# octet.
#
# <table>
# <tr><th>Bitmask</th><th>Definition</th></tr>
# <tr><th><tt>0b00______</tt></th><td>Universal (ASN.1 Native) Types</td></tr>
# <tr><th><tt>0b01______</tt></th><td>Application Types</td></tr>
# <tr><th><tt>0b10______</tt></th><td>Context-Specific Types</td></tr>
# <tr><th><tt>0b11______</tt></th><td>Private Types</td></tr>
# </table>
TAG_CLASS = {
:universal => 0b00000000, # 0
:application => 0b01000000, # 64
:context_specific => 0b10000000, # 128
:private => 0b11000000, # 192
}
##
# BER encoding type is kept in bit 6 of the tag type octet.
#
# <table>
# <tr><th>Bitmask</th><th>Definition</th></tr>
# <tr><th><tt>0b__0_____</tt></th><td>Primitive</td></tr>
# <tr><th><tt>0b__1_____</tt></th><td>Constructed</td></tr>
# </table>
ENCODING_TYPE = {
:primitive => 0b00000000, # 0
:constructed => 0b00100000, # 32
}
##
# Accepts a hash of hashes describing a BER syntax and converts it into
# a byte-keyed object for fast BER conversion lookup. The resulting
# "compiled" syntax is used by Net::BER::BERParser.
#
# This method should be called only by client classes of Net::BER (e.g.,
# Net::LDAP and Net::SNMP) and not by clients of those classes.
#
# The hash-based syntax uses TAG_CLASS keys that contain hashes of
# ENCODING_TYPE keys that contain tag numbers with object type markers.
#
# :<TAG_CLASS> => {
# :<ENCODING_TYPE> => {
# <number> => <object-type>
# },
# },
#
# === Permitted Object Types
# <tt>:string</tt>:: A string value, represented as BerIdentifiedString.
# <tt>:integer</tt>:: An integer value, represented with Fixnum.
# <tt>:oid</tt>:: An Object Identifier value; see X.690 section
# 8.19. Currently represented with a standard array,
# but may be better represented as a
# BerIdentifiedOID object.
# <tt>:array</tt>:: A sequence, represented as BerIdentifiedArray.
# <tt>:boolean</tt>:: A boolean value, represented as +true+ or +false+.
# <tt>:null</tt>:: A null value, represented as BerIdentifiedNull.
#
# === Example
# Net::LDAP defines its ASN.1 BER syntax something like this:
#
# class Net::LDAP
# AsnSyntax = Net::BER.compile_syntax({
# :application => {
# :primitive => {
# 2 => :null,
# },
# :constructed => {
# 0 => :array,
# # ...
# },
# },
# :context_specific => {
# :primitive => {
# 0 => :string,
# # ...
# },
# :constructed => {
# 0 => :array,
# # ...
# },
# }
# })
# end
#
# NOTE:: For readability and formatting purposes, Net::LDAP and its
# siblings actually construct their syntaxes more deliberately,
# as shown below. Since a hash is passed in the end in any case,
# the format does not matter.
#
# primitive = { 2 => :null }
# constructed = {
# 0 => :array,
# # ...
# }
# application = {
# :primitive => primitive,
# :constructed => constructed
# }
#
# primitive = {
# 0 => :string,
# # ...
# }
# constructed = {
# 0 => :array,
# # ...
# }
# context_specific = {
# :primitive => primitive,
# :constructed => constructed
# }
# AsnSyntax = Net::BER.compile_syntax(:application => application,
# :context_specific => context_specific)
def self.compile_syntax(syntax)
# TODO 20100327 AZ: Should we be allocating an array of 256 values
# that will either be +nil+ or an object type symbol, or should we
# allocate an empty Hash since unknown values return +nil+ anyway?
out = [ nil ] * 256
syntax.each do |tag_class_id, encodings|
tag_class = TAG_CLASS[tag_class_id]
encodings.each do |encoding_id, classes|
encoding = ENCODING_TYPE[encoding_id]
object_class = tag_class + encoding
classes.each do |number, object_type|
out[object_class + number] = object_type
end
end
end
out
end
end
end
class Net::BER::BerError < RuntimeError; end
##
# An Array object with a BER identifier attached.
class Net::BER::BerIdentifiedArray < Array
attr_accessor :ber_identifier
def initialize(*args)
super
end
end
##
# A BER object identifier.
class Net::BER::BerIdentifiedOid
attr_accessor :ber_identifier
def initialize(oid)
if oid.is_a?(String)
oid = oid.split(/\./).map {|s| s.to_i }
end
@value = oid
end
def to_ber
to_ber_oid
end
def to_ber_oid
@value.to_ber_oid
end
def to_s
@value.join(".")
end
def to_arr
@value.dup
end
end
##
# A String object with a BER identifier attached.
class Net::BER::BerIdentifiedString < String
attr_accessor :ber_identifier
def initialize args
super args
end
end
module Net::BER
##
# A BER null object.
class BerIdentifiedNull
attr_accessor :ber_identifier
def to_ber
"\005\000"
end
end
##
# The default BerIdentifiedNull object.
Null = Net::BER::BerIdentifiedNull.new
end
require 'net/ber/core_ext'

View File

@ -1,112 +1,168 @@
require 'stringio'
module Net
module BER
module BERParser
VERSION = '0.1.0'
##
# Implements Basic Encoding Rules parsing to be mixed into types as needed.
module Net::BER::BERParser
primitive = {
1 => :boolean,
2 => :integer,
4 => :string,
5 => :null,
6 => :oid,
10 => :integer,
13 => :string # (relative OID)
}
constructed = {
16 => :array,
17 => :array
}
universal = { :primitive => primitive, :constructed => constructed }
# The order of these follows the class-codes in BER.
# Maybe this should have been a hash.
TagClasses = [:universal, :application, :context_specific, :private]
primitive = { 10 => :integer }
context = { :primitive => primitive }
BuiltinSyntax = Net::BER.compile_syntax( {
:universal => {
:primitive => {
1 => :boolean,
2 => :integer,
4 => :string,
5 => :null,
6 => :oid,
10 => :integer,
13 => :string # (relative OID)
},
:constructed => {
16 => :array,
17 => :array
}
},
:context_specific => {
:primitive => {
10 => :integer
}
}
})
# The universal, built-in ASN.1 BER syntax.
BuiltinSyntax = Net::BER.compile_syntax(:universal => universal,
:context_specific => context)
def read_ber syntax=nil
# TODO: clean this up so it works properly with partial
# packets coming from streams that don't block when
# we ask for more data (like StringIOs). At it is,
# this can throw TypeErrors and other nasties.
id = getbyte or return nil # don't trash this value, we'll use it later
##
# This is an extract of our BER object parsing to simplify our
# understanding of how we parse basic BER object types.
def parse_ber_object(syntax, id, data)
# Find the object type from either the provided syntax lookup table or
# the built-in syntax lookup table.
#
# This exceptionally clever bit of code is verrrry slow.
object_type = (syntax && syntax[id]) || BuiltinSyntax[id]
n = getbyte
lengthlength,contentlength = if n <= 127
[1,n]
else
# Replaced the inject because it profiles hot.
# j = (0...(n & 127)).inject(0) {|mem,x| mem = (mem << 8) + getc}
j = 0
read( n & 127 ).each_byte {|n1| j = (j << 8) + n1}
[1 + (n & 127), j]
end
newobj = read contentlength
# This exceptionally clever and clear bit of code is verrrry slow.
objtype = (syntax && syntax[id]) || BuiltinSyntax[id]
# == is expensive so sort this if/else so the common cases are at the top.
obj = if objtype == :string
#(newobj || "").dup
s = BerIdentifiedString.new( newobj || "" )
s.ber_identifier = id
s
elsif objtype == :integer
j = 0
newobj.each_byte {|b| j = (j << 8) + b}
j
elsif objtype == :oid
# cf X.690 pgh 8.19 for an explanation of this algorithm.
# Potentially not good enough. We may need a BerIdentifiedOid
# as a subclass of BerIdentifiedArray, to get the ber identifier
# and also a to_s method that produces the familiar dotted notation.
oid = newobj.unpack("w*")
f = oid.shift
g = if f < 40
# == is expensive so sort this so the common cases are at the top.
if object_type == :string
s = Net::BER::BerIdentifiedString.new(data || "")
s.ber_identifier = id
s
elsif object_type == :integer
j = 0
data.each_byte { |b| j = (j << 8) + b }
j
elsif object_type == :oid
# See X.690 pgh 8.19 for an explanation of this algorithm.
# This is potentially not good enough. We may need a
# BerIdentifiedOid as a subclass of BerIdentifiedArray, to
# get the ber identifier and also a to_s method that produces
# the familiar dotted notation.
oid = data.unpack("w*")
f = oid.shift
g = if f < 40
[0, f]
elsif f < 80
[1, f-40]
[1, f - 40]
else
[2, f-80] # f-80 can easily be > 80. What a weird optimization.
# f - 80 can easily be > 80. What a weird optimization.
[2, f - 80]
end
oid.unshift g.last
oid.unshift g.first
oid
elsif objtype == :array
#seq = []
seq = BerIdentifiedArray.new
seq.ber_identifier = id
sio = StringIO.new( newobj || "" )
# Interpret the subobject, but note how the loop
# is built: nil ends the loop, but false (a valid
# BER value) does not!
while (e = sio.read_ber(syntax)) != nil
seq << e
end
seq
elsif objtype == :boolean
newobj != "\000"
elsif objtype == :null
n = BerIdentifiedNull.new
n.ber_identifier = id
n
else
raise BerError, "unsupported object type: id=0x#{id.to_s(16)}"
end
obj
oid.unshift g.last
oid.unshift g.first
# Net::BER::BerIdentifiedOid.new(oid)
oid
elsif object_type == :array
seq = Net::BER::BerIdentifiedArray.new
seq.ber_identifier = id
sio = StringIO.new(data || "")
# Interpret the subobject, but note how the loop is built:
# nil ends the loop, but false (a valid BER value) does not!
while (e = sio.read_ber(syntax)) != nil
seq << e
end
seq
elsif object_type == :boolean
data != "\000"
elsif object_type == :null
n = Net::BER::BerIdentifiedNull.new
n.ber_identifier = id
n
else
raise Net::BER::BerError, "Unsupported object type: id=#{id}"
end
end
private :parse_ber_object
##
# This is an extract of how our BER object length parsing is done to
# simplify the primary call. This is defined in X.690 section 8.1.3.
#
# The BER length will either be a single byte or up to 126 bytes in
# length. There is a special case of a BER length indicating that the
# content-length is undefined and will be identified by the presence of
# two null values (0x00 0x00).
#
# <table>
# <tr>
# <th>Range</th>
# <th>Length</th>
# </tr>
# <tr>
# <th>0x00 -- 0x7f<br />0b00000000 -- 0b01111111</th>
# <td>0 - 127 bytes</td>
# </tr>
# <tr>
# <th>0x80<br />0b10000000</th>
# <td>Indeterminate (end-of-content marker required)</td>
# </tr>
# <tr>
# <th>0x81 -- 0xfe<br />0b10000001 -- 0b11111110</th>
# <td>1 - 126 bytes of length as an integer value</td>
# </tr>
# <tr>
# <th>0xff<br />0b11111111</th>
# <td>Illegal (reserved for future expansion)</td>
# </tr>
# </table>
#
#--
# This has been modified from the version that was previously inside
# #read_ber to handle both the indeterminate terminator case and the
# invalid BER length case. Because the "lengthlength" value was not used
# inside of #read_ber, we no longer return it.
def read_ber_length
n = getbyte
if n <= 0x7f
n
elsif n == 0x80
-1
elsif n == 0xff
raise Net::BER::BerError, "Invalid BER length 0xFF detected."
else
v = 0
read(n & 0x7f).each_byte do |b|
v = (v << 8) + b
end
v
end
end
private :read_ber_length
##
# Reads a BER object from the including object. Requires that #getbyte is
# implemented on the including object and that it returns a Fixnum value.
# Also requires #read(bytes) to work.
#
# This does not work with non-blocking I/O.
def read_ber(syntax = nil)
# TODO: clean this up so it works properly with partial packets coming
# from streams that don't block when we ask for more data (like
# StringIOs). At it is, this can throw TypeErrors and other nasties.
id = getbyte or return nil # don't trash this value, we'll use it later
content_length = read_ber_length
if -1 == content_length
raise Net::BER::BerError, "Indeterminite BER content length not implemented."
else
data = read(content_length)
end
parse_ber_object(syntax, id, data)
end
end