diff --git a/README.md b/README.md index 0b359ae..df11419 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # sierra-postgres-utilities -Ruby connection to iii Sierra ILS postgres database / SierraDNA, meant to simplify making queries, exporting results, and some lookup / manipulation / transformation of MARC or non-MARC Sierra data and records. +Ruby connection and ORM for iii Sierra ILS postgres database / SierraDNA, meant to simplify making querying, manipulating, and exporting MARC or non-MARC Sierra data and records. __NOTE: This is in early development and future changes may well not be backwards compatible.__ @@ -13,24 +13,34 @@ __NOTE: Some sites may have iii setups that store different data in different pl ```ruby require 'sierra_postgres_utilities' +# Retrieve record by bnum/rnum bnum = 'b9256886a' -bib = SierraBib.new(bnum) +bib = Sierra::Record.get(bnum) + +# or retrieve by id +bib = Sierra::Record.get(id: 420916051894) bib.suppressed? #=> false bib.deleted? #=> false bib.mat_type #=> "a" -# Get data from sierra_view.bib_record as a hash -bib.bib_record #=> {:id=>420916051894, +# Get data from record_metadata and bib_record/item_record/etc. as a hash +bib.values #=> {:id=>420916051894, + # :record_num=>9256886 # :record_id=>420916051894, - # :language_code=>"eng", + # :creation_date_gmt=>2018-07-24 16:02:39 -0400, + # :deletion_date_gmt=>nil, + # ... # :bcode1=>"m", # :bcode2=>"a", # :bcode3=>"-", # .... # :is_suppressed=>false} -# Get bib as a ruby-marc object (https://github.com/ruby-marc/ruby-marc/) +# All of those hash keys are also available as methods +bib.bcode1 #=> "m" + +# Get rec's MARC as a ruby-marc object (https://github.com/ruby-marc/ruby-marc/) bib.marc # Write MARC to binary file (as per normal ruby-marc) @@ -51,7 +61,7 @@ puts bib.marc.to_mrk # ... # Get an array of item records attached to the bib -bib.items #=> [#] +bib.items #=> [#] item = bib.items.first item.status_code #=> "-" @@ -59,26 +69,50 @@ item.status_description #=> "Available" item.barcodes #=> ["00053203834"] ``` +### Sequel Datasets, Associations, and Querying + +Many record-types, fields, properties (e.g. itype) have Sequel models and associations available under Sierra::Data (though some have not been implemented due to lack of local relevance or use). + +See for Sequel documentation + +```ruby + +b = Sierra::Data::Bib.first #=> # # [{"record_id"=>"425206113029", "varfield_id"=>"57093780", ...}] +Sierra::DB.results.sql -SierraDB.results.values # results as array of record arrays - #=> [["425206113029", "57093780", ...]] +Sierra::DB.results.all # results as array of record hashes + #=> [{"record_id"=>"425206113029", "varfield_id"=>"57093780", ...}] # Write results to file -SierraDB.write_results('output.tsv') -SierraDB.write_results('output.csv', format: 'csv') -SierraDB.write_results('output.xlsx', format: 'xlsx') # Windows only. Maybe Mac. Not Linux. +Sierra::DB.write_results('output.tsv') +Sierra::DB.write_results('output.csv', format: 'csv') +Sierra::DB.write_results('output.xlsx', format: 'xlsx') # Requires WIN32OLE so probably Windows-only. # Send results as attachment details = {:from => 'user@example.com', @@ -86,32 +120,29 @@ details = {:from => 'user@example.com', :cc => 'also@example.com', :subject => 'that query', :body => 'Attached.'} -SierraDB.mail_results('output.tsv', mail_details: details) +Sierra::DB.mail_results('output.tsv', mail_details: details) ``` ### Retrieve arbitrary views -Retrieve arbitrary views as arrays of OpenStruct objects via ```SierraDB.[view_name]```. + +Retrieve arbitrary views as arrays of hashes via ```Sierra::DB.db[:view_name]```. + ```ruby -SierraDB.item_status_property_myuser. - map { |r| [r.code, r.name] }. - to_h - #=> {"!"=>"ON HOLDSHELF", "$"=>"LOST AND PAID", ... +Sierra::DB.db[:request_rule].first + #=> {:id=>6265, :record_type_code=>"i", :query ... ``` -### Retrieve defined views in the context of a particular record -Retrieve records related to a specific object via object.[view_name]. E.g.: -```bib.bib_record_item_record_link``` and ```bib.bib_record_property``` -return any of ```bib```'s entries in those two views. - ## SETUP -* git clone https://github.com/UNC-Libraries/sierra-postgres-utilities +* git clone * cd sierra-postgres-utilities * bundle install * bundle exec rake install * supply the Sierra postgres credentials per the below * optionally supply smtp server address +When possible, it is recommended that you also install the ```sequel_pg``` gem which makes database access significantly faster. See for installation details / requirements. + ### Credentials Create a yaml file in the base directory like so: @@ -142,11 +173,6 @@ ENV['SIERRA_INIT_CREDS'] = 'my/path/file.yaml' require 'sierra_postgres_utilities' ``` -Once connected to the Sierra DB, you can close the connection and reconnect -under alternate creds using: -- ```SierraDB.connect_as(creds: filename)```, or -- ```SierraDB.connect_as(creds: cred_hash)``` - ### SMTP connection / email address storage Define an smtp connection (that does not require authentication) if you'll use this to send emails. @@ -156,18 +182,3 @@ Create ```smtp.secret``` in the working directory: address: smtp.example.com port: 25 ``` - -You can create a YAML file ```email.secret```. Example contents: - -```yaml -default_email: user@example.com -other_email: other_user@example.com -``` - -And then in ruby: - -```ruby -c.yield_email # => user@example.com -c.yield_email(index: 'default_email') # => user@example.com -c.yield_email(index: 'other_email') # => other_user@example.com -``` diff --git a/ext/marc/controlfield.rb b/lib/ext_spu/marc/controlfield.rb similarity index 89% rename from ext/marc/controlfield.rb rename to lib/ext_spu/marc/controlfield.rb index 1a56191..90bd8ec 100644 --- a/ext/marc/controlfield.rb +++ b/lib/ext_spu/marc/controlfield.rb @@ -1,6 +1,9 @@ +require_relative 'xml_helper' + module MARC - # Extend the MARC::Controlfield class with some UNC-specific helpers + # Extends MARC::Controlfield. class ControlField + include MARC::XMLHelper # Convert field to a mrk-type string # e.g. ControlField.new('001', 'ocm12345').to_mrk @@ -10,6 +13,11 @@ def to_mrk "=#{tag} #{value}" end + def xml_string(_) + data = escape_xml_reserved(value) + " #{data}" + end + # Returns Sierra style content string. # It's less silly than this for DataFields, but useful to also define # here so we can call field_content on any field. diff --git a/ext/marc/datafield.rb b/lib/ext_spu/marc/datafield.rb similarity index 90% rename from ext/marc/datafield.rb rename to lib/ext_spu/marc/datafield.rb index 7866854..5e3a00a 100644 --- a/ext/marc/datafield.rb +++ b/lib/ext_spu/marc/datafield.rb @@ -1,6 +1,9 @@ +require_relative 'xml_helper' + module MARC - #Extend the MARC::Datafield class with some UNC-specific helpers + # Extends MARC::Datafield. class DataField + include MARC::XMLHelper def add_subfields!(sf_tag, new_data) # add a string or array of string to datafield as subfields @@ -9,7 +12,7 @@ def add_subfields!(sf_tag, new_data) new_data = [new_data] unless new_data.is_a?(Array) new_data.each do |sf_data| next unless sf_data - sf = MARC::Subfield.new(sf_tag, sf_data ) + sf = MARC::Subfield.new(sf_tag, sf_data) self.append(sf) end self @@ -52,6 +55,20 @@ def field_content f end + def xml_string(strip_datafields: true) + return if subfields.empty? + xml = '' + xml << " \n" + subfields.each do |sf| + data = escape_xml_reserved(sf.value) + data.strip! if strip_datafields + xml << " #{data}\n" + end + xml << ' ' + xml + end + # Filters subfields based on subfield code/value criteria. # Returns subfields that equal / dont' equal # strings, or match / don't match regexps @@ -167,7 +184,7 @@ def meets_criteria?(tag: nil, ind1: nil, ind2: nil, value: nil, # remove candidates according to complex subfield criteria complex_subfields.each do |rule| unless rule.is_a?(Array) - raise "complex_subfields should be an array of arrays." + raise 'complex_subfields should be an array of arrays.' end type, hsh = rule case type diff --git a/ext/marc/record.rb b/lib/ext_spu/marc/record.rb similarity index 84% rename from ext/marc/record.rb rename to lib/ext_spu/marc/record.rb index 0a299dd..4cf368f 100644 --- a/ext/marc/record.rb +++ b/lib/ext_spu/marc/record.rb @@ -1,271 +1,304 @@ -module MARC - # Extend the MARC::Record class with some UNC-specific helpers - - XML_HEADER = <<~XML.freeze - - - XML - XML_FOOTER = ''.freeze - - class Record - attr_reader :oclcnum - - # lazy evaluated - @lang_code_map = nil - - def to_mrk - mrk = '' - mrk += "=LDR #{leader}\n" if leader - fields.each do |f| - mrk += "#{f.to_mrk}\n" - end - mrk - end - - def oclcnum - @oclcnum ||= get_oclcnum - end - - def get_oclcnum - @oclcnum = nil # prevents using previous value if re-deriving - oclcnum_003s = ['', 'OCoLC', 'NhCcYBP'] - my001 = self['001']&.value&.strip.to_s - my003 = self['003']&.value&.strip.to_s - - # if appropriate 003, try for digits with optional hsl/tmp prefix - if oclcnum_003s.include?(my003) - m = my001.match(/^(?:hsl|tmp)?(\d+)$/) - if m - return @oclcnum = m[1] - end - end - - # try for digits followed by alphanumeric suffix - m = my001.match(/^(\d+)\D\w+$/) - if m - @oclcnum = m[1] - else # look in the 035 - m035oclcnums = get_035oclcnums - @oclcnum = m035oclcnums&.first - end - @oclcnum - end # def get_oclcnum - - def get_035oclcnums - m035oclcnums = nil - oclc035s = fields('035'). - map { |f| f.subfield_search(code: 'a', value: /^\(OCoLC\)/) }. - flatten. - map(&:value) - return nil if oclc035s.empty? - oclc035s.map! { |v| v.gsub(/\(OCoLC\)0*/, '') } - oclc035s.reject! { |v| v.match(/^M-ESTCN/) } - oclc035s.map! { |v| v.gsub(/ocn|ocm|on/, '') } - oclc035s.reject! { |v| v == '' } - m035oclcnums = oclc035s unless oclc035s.empty? - m035oclcnums - end # def get_035oclcnums - - def no_leader? - return true unless leader && !leader.empty? - end - - def bad_leader_length? - return nil if no_leader? - return true if leader.length != 24 - end - - # Is rec type missing or invalid? - # True when no/empty leader or when ldr06 is not an allowed code - def ldr06_invalid? - return true unless leader - leader[6] !~ /[acdefgijkmoprt]/ - end - - # Is blvl missing or invalid? - # True when no/empty leader or when ldr07 is not an allowed code - def ldr07_invalid? - return true unless leader - leader[7] !~ /[abcdims]/ - end - - def bad_008_length? - # true if any 008 length != 40 - # This check to make sure the 008 is 40 chars long. Afaik Sierra postgres - # would store an 008 of just "a" and an 008 of "a [...40 chars]" - # exactly the same. We'd retrieve both as "a" followed by 39 spaces. - # We're already checking for a valid language code in 008/35-37 - # and 008/38-39 don't need to be non-blank. So whether this check - # has any added value seems questionable. - my008s = find_all { |f| f.tag == '008' } - return nil unless my008s - my008s.reject! { |f| f.value.length == 40 } - return true unless my008s.empty? - end - - # returns [008/35-37, full language name] - # if invalid language code, returns [008/35-37, nil] - # if no 008, returns nil - # forbid_discontinued treats discontinued language codes - # as invalid - def language_from_008(forbid_discontinued: false) - code = self['008']&.value&.slice(35..37) - return nil unless code - language = self.class.lang_code_map[code] - - # try for a discontinued language code when needed and - # allowed - unless language || forbid_discontinued - language = self.class.lang_code_map["-#{code}"] - end - - [code, language] - end - - # Sets lang_code_map once requested - def self.lang_code_map - @lang_code_map ||= YAML.load_file( - File.join(__dir__, '../../data/marc_language_codes.yml') - ) - end - - # true when no 245s or no 245 contains an $a or $k - # false when >=1 245 has >=1 $a or $k - def no_245_has_ak? - no_fields?(tag: '245', complex_subfields: [[:has, code: /[ak]/]]) - end - - def count(tag) - # counts number of fields for given marc tag - fields = find_all { |f| f.tag == tag } - return nil unless fields - fields.length - end - - # true if there is >=1 300 fields without a $a - # false if there are no 300 fields, or every 300 has a $a - def m300_without_a? - any_fields?(tag: '300', complex_subfields: [[:has_no, code: 'a']]) - end - - def oclc_035_count - m035oclcnums = get_035oclcnums - return 0 unless m035oclcnums - m035oclcnums.length - end - - def m035_lacks_oclcnum? - # true if 035 lacks sierra oclcnum (e.g. from 001, 035) - # even if 035 has some other oclcnum - return false unless oclcnum - my035oclcnums = get_035oclcnums - return true unless my035oclcnums - return false if my035oclcnums.include?(@oclcnum) - true - end - - # sorts marc record by tag - # ordering of fields with the same tag is retained - def sort - sorter = to_hash - sorter['fields'] = - sorter['fields'].sort_by.with_index { |f, idx| [f.keys, idx] } - MARC::Record.new_from_hash(sorter) - end - - # Filters fields based on criteria. - # Returns empty array when no matches - # When criteria is a string, properties must equal / not equal the string. - # When criteria is a regexp, properties must match / not match the regexp. - # For example: - # Select 500 fields (same as rec.fields('500') ) - # field_find_all(tag: '500') - # Select 856s with ind1 != 4, where content includes "http" - # field_find_all(tag: '856', ind1_not: '4', content: /http/ ) - # - # Complex_subfields uses the subfield_search in DataField extension and - # :has, :has_no, :has_one, :has_as_first parameters. - # has some $a - # [:has, code: 'a'}, - # has no $b - # [:has_no, code: 'b'}, - # has only $b(s) - # [:has_no, code_not: 'b'}, - # has only sf_contents = 'not_useful' - # [:has_no, value_not: 'not_useful'}, - # has at least one $a or $k that matches /foo/ - # [:has, code: /[ak]/, value: /foo/}, - # has at least one $a != bar - # [:has, code: 'a', value_not: 'bar'}, - # does not have any $a whose content is not foo - # [:has_no, code: 'a', value_not: 'foo'}, - # does not have any $b whose content is bar - # [:has_no, code: 'b', value: 'bar'}, - # has some non-$b subfield that equals foo - # [:has, code_not: 'b', value: 'foo'}, - # has some non$b-subfield whose content is not bar - # [:has, code_not: 'b', value_not: 'bar'}, - # has no non$b-subfields whose content is foo - # [:has_no, code_not: /b/, value: 'foo'}, - # has no non$b-subfields whose content is not bar - # [:has_no, code_not: /b/, value_not: 'bar'} - # 245s with: - # >=1 $a containing foo - # >=1 subfield that isn't $c and does equal 'foo' - # 0 $b that do not contain bar - # exactly 1 $h - # field_find_all(tag: '245', complex_subfields: [ - # [:has, code: 'a', value: /foo/], - # [:has, code_not: 'c', value: 'foo'] - # [:has_no, code: 'b', value_not: /bar/], - # [:has_one, code: 'h'] - # ]) - # - # A simple subfield criteria mirroring the tag/ind1/etc. criteria - # is not included because value as a field_find_all keyword refers to the - # value of the entire field, and it seems easy to mistake - # field_find_all(subfield: 'a', value: /foo/) - # for something that is looking for foo inside a $a when it would - # be looking for a field with a subfield 'a' where the field's value - # contained foo. - # - def field_find_all(**args) - # if looking for a specific field, use built-in hash for initial filter, - # otherwise get all fields. - fields_in_scope = - if args[:tag].is_a?(String) - fields(args[:tag]).dup - else - @fields.dup - end - fields_in_scope.select { |f| f.meets_criteria?(args) } - end - - # Returns first field that matches criteria - # (see field_find_all for arguments) - def field_find(**args) - # if looking for a specific field, use built-in hash for initial filter, - # otherwise get all fields. - fields_in_scope = - if args[:tag].is_a?(String) - fields(args[:tag]).dup - else - @fields.dup - end - fields_in_scope.each { |f| return f if f.meets_criteria?(args) } - nil - end - - def any_fields?(**args) - !!field_find(args) - end - - def one_field?(**args) - field_find_all(args).count == 1 - end - - def no_fields?(**args) - field_find(args).nil? - end - end # class Record -end # module MARC +require 'yaml' +require_relative 'xml_helper' + +module MARC + # Extends MARC::Record with, sometimes UNC-specific, helpers. + class Record + include MARC::XMLHelper + + # lazy evaluated + @lang_code_map = nil + + def to_mrk + mrk = '' + mrk += "=LDR #{leader}\n" if leader + fields.each do |f| + mrk += "#{f.to_mrk}\n" + end + mrk + end + + def xml_string(strip_datafields: true) + MARC::Record.xml_string(self, strip_datafields: strip_datafields) + end + + # Returns a string of xml with "sensible" whitespacing. + # whitespace in text nodes retained + # linebreaks added to make human readable + # I believe options for in-built readers we tried were + # either/or in those areas. + # + # Writes the MARC faithfully, except: + # datafields (not controlfields) are stripped of leading/trailing + # whitespace (default) + # drops any 002/004/009 fields + # drops any datafields containing no subfields + # xml escapes reserved characters + def self.xml_string(marc, strip_datafields: true) + xml = '' + xml << "\n" + xml << " #{marc.leader}\n" if marc.leader + + # Skip writing xml for invalid control fields (002, 004, 009) or for + # datafields where no subfield exists. + # Note: This is not skipping data fields with >= a single empty subfield + # e.g. not skipping "=856 42|u" + # This is skipping fields with no subfield + # e.g. skipping "=856 42|" and "=856 42" + marc.reject { |f| + f.tag =~ /00[249]$/ || (f.tag =~ /^0[1-9]/ && f.subfields.empty?) + }.each do |f| + xml << "#{f.xml_string(strip_datafields: strip_datafields)}\n" + end + + xml << "\n" + xml + end + + def oclcnum + @oclcnum ||= get_oclcnum + end + + def get_oclcnum + @oclcnum = nil # prevents using previous value if re-deriving + oclcnum_003s = ['', 'OCoLC', 'NhCcYBP'] + my001 = self['001']&.value&.strip.to_s + my003 = self['003']&.value&.strip.to_s + + # if appropriate 003, try for digits with optional hsl/tmp prefix + if oclcnum_003s.include?(my003) + m = my001.match(/^(?:hsl|tmp)?(\d+)$/) + if m + return @oclcnum = m[1] + end + end + + # try for digits followed by alphanumeric suffix + m = my001.match(/^(\d+)\D\w+$/) + if m + @oclcnum = m[1] + else # look in the 035 + m035oclcnums = get_035oclcnums + @oclcnum = m035oclcnums&.first + end + @oclcnum + end # def get_oclcnum + + def get_035oclcnums + m035oclcnums = nil + oclc035s = fields('035'). + map { |f| f.subfield_search(code: 'a', value: /^\(OCoLC\)/) }. + flatten. + map(&:value) + return nil if oclc035s.empty? + oclc035s.map! { |v| v.gsub(/\(OCoLC\)0*/, '') } + oclc035s.reject! { |v| v.match(/^M-ESTCN/) } + oclc035s.map! { |v| v.gsub(/ocn|ocm|on/, '') } + oclc035s.reject! { |v| v == '' } + m035oclcnums = oclc035s unless oclc035s.empty? + m035oclcnums + end # def get_035oclcnums + + def no_leader? + return true unless leader && !leader.empty? + end + + def bad_leader_length? + return nil if no_leader? + return true if leader.length != 24 + end + + # Is rec type missing or invalid? + # True when no/empty leader or when ldr06 is not an allowed code + def ldr06_invalid? + return true unless leader + leader[6] !~ /[acdefgijkmoprt]/ + end + + # Is blvl missing or invalid? + # True when no/empty leader or when ldr07 is not an allowed code + def ldr07_invalid? + return true unless leader + leader[7] !~ /[abcdims]/ + end + + def bad_008_length? + # true if any 008 length != 40 + # This check to make sure the 008 is 40 chars long. Afaik Sierra postgres + # would store an 008 of just "a" and an 008 of "a [...40 chars]" + # exactly the same. We'd retrieve both as "a" followed by 39 spaces. + # We're already checking for a valid language code in 008/35-37 + # and 008/38-39 don't need to be non-blank. So whether this check + # has any added value seems questionable. + my008s = find_all { |f| f.tag == '008' } + return nil unless my008s + my008s.reject! { |f| f.value.length == 40 } + return true unless my008s.empty? + end + + # returns [008/35-37, full language name] + # if invalid language code, returns [008/35-37, nil] + # if no 008, returns nil + # forbid_discontinued treats discontinued language codes + # as invalid + def language_from_008(forbid_discontinued: false) + code = self['008']&.value&.slice(35..37) + return nil unless code + language = self.class.lang_code_map[code] + + # try for a discontinued language code when needed and + # allowed + unless language || forbid_discontinued + language = self.class.lang_code_map["-#{code}"] + end + + [code, language] + end + + # Sets lang_code_map once requested + def self.lang_code_map + @lang_code_map ||= YAML.load_file( + File.join(__dir__, '../../../data/marc_language_codes.yml') + ) + end + + # true when no 245s or no 245 contains an $a or $k + # false when >=1 245 has >=1 $a or $k + def no_245_has_ak? + no_fields?(tag: '245', complex_subfields: [[:has, code: /[ak]/]]) + end + + def count(tag) + # counts number of fields for given marc tag + fields = find_all { |f| f.tag == tag } + return nil unless fields + fields.length + end + + # true if there is >=1 300 fields without a $a + # false if there are no 300 fields, or every 300 has a $a + def m300_without_a? + any_fields?(tag: '300', complex_subfields: [[:has_no, code: 'a']]) + end + + def oclc_035_count + m035oclcnums = get_035oclcnums + return 0 unless m035oclcnums + m035oclcnums.length + end + + def m035_lacks_oclcnum? + # true if 035 lacks sierra oclcnum (e.g. from 001, 035) + # even if 035 has some other oclcnum + return false unless oclcnum + my035oclcnums = get_035oclcnums + return true unless my035oclcnums + return false if my035oclcnums.include?(@oclcnum) + true + end + + # sorts marc record by tag + # ordering of fields with the same tag is retained + def sort + sorter = to_hash + sorter['fields'] = + sorter['fields'].sort_by.with_index { |f, idx| [f.keys, idx] } + MARC::Record.new_from_hash(sorter) + end + + # Filters fields based on criteria. + # Returns empty array when no matches + # When criteria is a string, properties must equal / not equal the string. + # When criteria is a regexp, properties must match / not match the regexp. + # For example: + # Select 500 fields (same as rec.fields('500') ) + # field_find_all(tag: '500') + # Select 856s with ind1 != 4, where content includes "http" + # field_find_all(tag: '856', ind1_not: '4', content: /http/ ) + # + # Complex_subfields uses the subfield_search in DataField extension and + # :has, :has_no, :has_one, :has_as_first parameters. + # has some $a + # [:has, code: 'a'}, + # has no $b + # [:has_no, code: 'b'}, + # has only $b(s) + # [:has_no, code_not: 'b'}, + # has only sf_contents = 'not_useful' + # [:has_no, value_not: 'not_useful'}, + # has at least one $a or $k that matches /foo/ + # [:has, code: /[ak]/, value: /foo/}, + # has at least one $a != bar + # [:has, code: 'a', value_not: 'bar'}, + # does not have any $a whose content is not foo + # [:has_no, code: 'a', value_not: 'foo'}, + # does not have any $b whose content is bar + # [:has_no, code: 'b', value: 'bar'}, + # has some non-$b subfield that equals foo + # [:has, code_not: 'b', value: 'foo'}, + # has some non$b-subfield whose content is not bar + # [:has, code_not: 'b', value_not: 'bar'}, + # has no non$b-subfields whose content is foo + # [:has_no, code_not: /b/, value: 'foo'}, + # has no non$b-subfields whose content is not bar + # [:has_no, code_not: /b/, value_not: 'bar'} + # 245s with: + # >=1 $a containing foo + # >=1 subfield that isn't $c and does equal 'foo' + # 0 $b that do not contain bar + # exactly 1 $h + # field_find_all(tag: '245', complex_subfields: [ + # [:has, code: 'a', value: /foo/], + # [:has, code_not: 'c', value: 'foo'] + # [:has_no, code: 'b', value_not: /bar/], + # [:has_one, code: 'h'] + # ]) + # + # A simple subfield criteria mirroring the tag/ind1/etc. criteria + # is not included because value as a field_find_all keyword refers to the + # value of the entire field, and it seems easy to mistake + # field_find_all(subfield: 'a', value: /foo/) + # for something that is looking for foo inside a $a when it would + # be looking for a field with a subfield 'a' where the field's value + # contained foo. + # + def field_find_all(**args) + # if looking for a specific field, use built-in hash for initial filter, + # otherwise get all fields. + fields_in_scope = + if args[:tag].is_a?(String) + fields(args[:tag]).dup + else + @fields.dup + end + fields_in_scope.select { |f| f.meets_criteria?(args) } + end + + # Returns first field that matches criteria + # (see field_find_all for arguments) + def field_find(**args) + # if looking for a specific field, use built-in hash for initial filter, + # otherwise get all fields. + fields_in_scope = + if args[:tag].is_a?(String) + fields(args[:tag]).dup + else + @fields.dup + end + fields_in_scope.each { |f| return f if f.meets_criteria?(args) } + nil + end + + def any_fields?(**args) + !!field_find(args) + end + + def one_field?(**args) + field_find_all(args).count == 1 + end + + def no_fields?(**args) + field_find(args).nil? + end + end # class Record +end # module MARC diff --git a/lib/ext_spu/marc/xml_helper.rb b/lib/ext_spu/marc/xml_helper.rb new file mode 100644 index 0000000..09123a1 --- /dev/null +++ b/lib/ext_spu/marc/xml_helper.rb @@ -0,0 +1,33 @@ +require 'marc' +module MARC + # Tools to add marc-xml headers/footers and escape xml. + module XMLHelper + # XML document header, to be followed with xml for each record(s) + HEADER = <<~XML.freeze + + + XML + + # XML document footer, to follow xml the set of record(s) + FOOTER = ''.freeze + + # XML-escapes ampersands, brackets, quotes in a string. + # + # @param [String] data + # @return [String] escaped copy of given string + def escape_xml_reserved(data) + XMLHelper.escape_xml_reserved(data) + end + + # (see #escape_xml_reserved) + def self.escape_xml_reserved(data) + return data unless data =~ /[<>&"']/ + data. + gsub('&', '&'). + gsub('<', '<'). + gsub('>', '>'). + gsub('"', '"'). + gsub("'", ''') + end + end +end diff --git a/lib/ext_spu/sequel/model/model.rb b/lib/ext_spu/sequel/model/model.rb new file mode 100644 index 0000000..5b27933 --- /dev/null +++ b/lib/ext_spu/sequel/model/model.rb @@ -0,0 +1,41 @@ +# Extends Sequel. +module Sequel + # Extends Sequel::Model. + class Model + # Creates a prepared statement to retrieve instance(s) of the model + # and a class method on the model to use that prepared statement. + # + # @param [Symbol] field the column on the target class/table to match + # against + # @param [Symbol] select_type the form the results should take. + # Perhaps here, most often :select or :first + # + # @see http://sequel.jeremyevans.net/rdoc/classes/Sequel/Dataset/PreparedStatementMethods.html#method-i-prepared_sql + # @param [Array] sorting optionally specify fields to order by + # @return void + # @example Create prepared statement / method to retrieve varfields for a + # record_id. + # Sierra::Data::Varfield.prepare_retrieval_by( + # :record_id, + # :select, + # sorting: %i[marc_tag varfield_type_code occ_num id] + # ) + def self.prepare_retrieval_by(field, select_type, sorting: nil) + class_name = name.split('::').last.downcase + statement_name = "#{class_name}_by_#{field}".to_sym + + if sorting + where(field => :"$#{field}"). + order(sorting). + prepare(select_type, statement_name) + else + where(field => :"$#{field}"). + prepare(select_type, statement_name) + end + + define_singleton_method :"by_#{field}" do |value| + Sierra::DB.db.call(statement_name, value) + end + end + end +end diff --git a/lib/sierra_postgres_utilities.rb b/lib/sierra_postgres_utilities.rb index c6a84f3..83b89f4 100644 --- a/lib/sierra_postgres_utilities.rb +++ b/lib/sierra_postgres_utilities.rb @@ -1,26 +1,35 @@ -require 'csv' -require 'yaml' -require 'mail' -require 'pg' +# Utilities to access, model, manipulate iii Sierra data from the iii +# Sierra postgres database. +module Sierra + require 'marc' -require 'marc' -require_relative '../ext/marc/record' -require_relative '../ext/marc/datafield' -require_relative '../ext/marc/controlfield' + require_relative 'sierra_postgres_utilities/logging' + require 'sequel' + require_relative 'ext_spu/sequel/model/model' + require_relative 'sierra_postgres_utilities/db' -require_relative 'sierra_postgres_utilities/sierradb' -# As it loads, sierra-postgres-utilities connects to the DB to prepare some -# queries, etc. Defining SIERRA_INIT_CREDS before loading sierra-postgres-utilities -# allows that initial connection to use the specified credentials -creds = ENV['SIERRA_INIT_CREDS'] || 'prod' -SierraDB.initial_creds(creds) + # Skip loading things that require a DB connection unless there is a + # working DB connection. If these are skipped, they will be loaded + # if/when a DB connection is established. + if Sierra::DB.connected? + require_relative 'sierra_postgres_utilities/data' -require_relative 'sierra_postgres_utilities/views' -require_relative 'sierra_postgres_utilities/helpers' -require_relative 'sierra_postgres_utilities/records' + require_relative 'sierra_postgres_utilities/search' + require_relative 'sierra_postgres_utilities/record' -require_relative 'sierra_postgres_utilities/hold' -require_relative 'sierra_postgres_utilities/user' -require_relative 'sierra_postgres_utilities/derivative_record' + require_relative 'sierra_postgres_utilities/derivative_bib' + end + + require_relative 'sierra_postgres_utilities/spec_support' + + # Some gems that also extend marc (e.g. marc-to-argot) only load paths + # not already in $LOAD_PATH. We name the 'ext' dir as 'ext_spu' so that + # 'ext/marc' will not already be in $LOAD_PATH, allowing both sets + # of extensions to load. + require_relative 'ext_spu/marc/xml_helper' + require_relative 'ext_spu/marc/controlfield' + require_relative 'ext_spu/marc/datafield' + require_relative 'ext_spu/marc/record' +end diff --git a/lib/sierra_postgres_utilities/data.rb b/lib/sierra_postgres_utilities/data.rb new file mode 100644 index 0000000..1c4f438 --- /dev/null +++ b/lib/sierra_postgres_utilities/data.rb @@ -0,0 +1,12 @@ +module Sierra + # Includes models for Sierra data, records, etc. + module Data + require_relative 'data/helpers' + + require_relative 'data/fields' + require_relative 'data/misc' + require_relative 'data/properties' + require_relative 'data/records' + require_relative 'data/secondary_records' + end +end diff --git a/lib/sierra_postgres_utilities/data/fields.rb b/lib/sierra_postgres_utilities/data/fields.rb new file mode 100644 index 0000000..fd60b6e --- /dev/null +++ b/lib/sierra_postgres_utilities/data/fields.rb @@ -0,0 +1,8 @@ +module Sierra + module Data + require_relative 'fields/control_field' + require_relative 'fields/leaderfield' + require_relative 'fields/subfield' + require_relative 'fields/varfield' + end +end diff --git a/lib/sierra_postgres_utilities/data/fields/control_field.rb b/lib/sierra_postgres_utilities/data/fields/control_field.rb new file mode 100644 index 0000000..d54d7bf --- /dev/null +++ b/lib/sierra_postgres_utilities/data/fields/control_field.rb @@ -0,0 +1,40 @@ +module Sierra + module Data + class ControlField < Sequel::Model(DB.db[:control_field]) + set_primary_key :id + prepare_retrieval_by :record_id, :select, + sorting: %i[varfield_type_code occ_num id] + + many_to_one :record_metadata, + class: :'Sierra::Data::Metadata', primary_key: :id, + key: :record_id + + many_to_one :bib, primary_key: :id, key: :record_id + many_to_one :item, primary_key: :id, key: :record_id + many_to_one :authority, primary_key: :id, key: :record_id + many_to_one :holdings, + class: :'Sierra::Data::Holdings', primary_key: :id, + key: :record_id + many_to_one :order, primary_key: :id, key: :record_id + many_to_one :patron, primary_key: :id, key: :record_id + + def record + record_metadata.record + end + + def to_s + value = to_hash.select { |k, _| k[/p\d+/] }. + values[0..39]. + map(&:to_s). + join + return value if control_num == 8 + return value[0..17] if control_num == 6 + return value.rstrip if control_num == 7 + end + + def to_marc + MARC::ControlField.new("00#{control_num}", to_s) + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/fields/leaderfield.rb b/lib/sierra_postgres_utilities/data/fields/leaderfield.rb new file mode 100644 index 0000000..09a3f82 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/fields/leaderfield.rb @@ -0,0 +1,47 @@ +module Sierra + module Data + class LeaderField < Sequel::Model(DB.db[:leader_field]) + set_primary_key :id + prepare_retrieval_by :record_id, :first + + many_to_one :record_metadata, + class: :'Sierra::Data::Metadata', primary_key: :id, + key: :record_id + + one_to_one :bib, primary_key: :id, key: :record_id + one_to_one :item, primary_key: :id, key: :record_id + one_to_one :authority, primary_key: :id, key: :record_id + one_to_one :holdings, + class: :'Sierra::Data::Holdings', primary_key: :id, + key: :record_id + one_to_one :order, primary_key: :id, key: :record_id + one_to_one :patron, primary_key: :id, key: :record_id + + def record + record_metadata.record + end + + def to_s + [ + '00000'.freeze, # rec_length + record_status_code, + record_type_code, + bib_level_code, + control_type_code, + char_encoding_scheme_code, + '2'.freeze, # indicator count + '2'.freeze, # subf_ct + base_address.to_s.rjust(5, '0'), + encoding_level_code, + descriptive_cat_form_code, + multipart_level_code, + '4500'.freeze # ldr_end + ].join + end + + def to_xml + " #{marc.leader}" + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/fields/subfield.rb b/lib/sierra_postgres_utilities/data/fields/subfield.rb new file mode 100644 index 0000000..e3e4be6 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/fields/subfield.rb @@ -0,0 +1,25 @@ +module Sierra + module Data + class Subfield < Sequel::Model(DB.db[:subfield]) + set_primary_key :id + + many_to_one :varfield, primary_key: :id, key: :varfield_id + many_to_one :record_metadata, + class: :'Sierra::Data::Metadata', primary_key: :id, + key: :record_id + + many_to_one :bib, primary_key: :id, key: :record_id + many_to_one :item, primary_key: :id, key: :record_id + many_to_one :authority, primary_key: :id, key: :record_id + many_to_one :holdings, + class: :'Sierra::Data::Holdings', primary_key: :id, + key: :record_id + many_to_one :order, primary_key: :id, key: :record_id + many_to_one :patron, primary_key: :id, key: :record_id + + def record + record_metadata.record + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/fields/varfield.rb b/lib/sierra_postgres_utilities/data/fields/varfield.rb new file mode 100644 index 0000000..120b616 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/fields/varfield.rb @@ -0,0 +1,75 @@ +require 'marc' + +module Sierra + module Data + class Varfield < Sequel::Model(DB.db[:varfield]) + set_primary_key :id + prepare_retrieval_by :record_id, :select, + sorting: %i[marc_tag varfield_type_code occ_num id] + + one_to_many :subfields, key: :varfield_id + many_to_one :record_metadata, + class: :'Sierra::Data::Metadata', primary_key: :id, + key: :record_id + + many_to_one :bib, primary_key: :id, key: :record_id + many_to_one :item, primary_key: :id, key: :record_id + many_to_one :authority, primary_key: :id, key: :record_id + many_to_one :holdings, + class: :'Sierra::Data::Holdings', primary_key: :id, + key: :record_id + many_to_one :order, primary_key: :id, key: :record_id + many_to_one :patron, primary_key: :id, key: :record_id + + def record + record_metadata.record + end + + def marc_varfield? + return true if marc_tag + false + end + + def nonmarc_varfield? + !marc_varfield? + end + + def control_field? + return true if marc_tag =~ /^00/ + false + end + + def to_marc + return unless marc_varfield? + if control_field? + MARC::ControlField.new(marc_tag, field_content) + else + MARC::DataField.new(marc_tag, marc_ind1, marc_ind2, + *Varfield.subfield_arry(field_content)) + end + end + + # Returns the first subfield with matching tag + def subfield(tag) + subfields.find { |sf| sf.tag == tag.to_s } + end + + def self.subfield_arry(field_content, implicit_sfa: true) + field_content = add_explicit_sf_a(field_content) if implicit_sfa + arry = field_content.split('|') + + # delete anything prior to the first subfield delimiter (which often + # but not always means deleting an empty string), then delete + # any/other empty strings + arry.shift + arry.delete(''.freeze) + arry.map { |x| [x[0], x[1..-1]] } + end + + def self.add_explicit_sf_a(field_content) + field_content = "|a#{field_content}" unless field_content.chr == '|' + field_content + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/helpers.rb b/lib/sierra_postgres_utilities/data/helpers.rb new file mode 100644 index 0000000..1ce3c4c --- /dev/null +++ b/lib/sierra_postgres_utilities/data/helpers.rb @@ -0,0 +1,6 @@ +module Sierra + module Data + require_relative 'helpers/phrase_normalization.rb' + require_relative 'helpers/sierra_marc.rb' + end +end diff --git a/lib/sierra_postgres_utilities/data/helpers/phrase_normalization.rb b/lib/sierra_postgres_utilities/data/helpers/phrase_normalization.rb new file mode 100644 index 0000000..5474eaf --- /dev/null +++ b/lib/sierra_postgres_utilities/data/helpers/phrase_normalization.rb @@ -0,0 +1,419 @@ +require 'English' +require 'i18n' +I18n.available_locales = [:en] + +# diacritics sometimes have multiple mapping entries. category 24, +# "standrule.unicode," may be the default, but not sure what rule really wins +# out. Here we sort the query so that category 24 mappings come last and +# overwrite any previously seen mappings when we make the hash. +sierra_mappings = + Sierra::DB.query( + "select diacritic, mapped_string from diacritic_mapping " \ + "order by diacritic_category_id = '24'" + ).to_a. + map(&:values). + to_h +I18n.backend.store_translations( + :en, + i18n: {transliterate: {rule: sierra_mappings}} +) + +module Sierra + module Data + module Helpers + # Methods that replicate Sierra's phrase_entry normalization functions + # for various index types. + # Actual Sierra documentation isn't specific enough to reconstruct + # Sierra's normalization. So this reconstruction is based on what + # Sierra seems to be doing and largely what it's doing to UNC's actual + # set of field. Because of this, there are surely edge cases + # where this normalization will not match Sierra's. For example, + # there are punctuation characters that we + + # Also, not much has been done beyond trying to get the normalizations + # to be correct. So, within normalization methods, expect that a more + # logical ordering of transformations than what we have is possible. + # Across normalization methods, assume that we may have separate + # methods for different indexes when Sierra may be using the same method + # (e.g. we have separate methods for bib utility numbers and "standard" + # normalization for titles/authors/etc. I'm unsure whether Sierra handles + # them differently. Our standard normalization is the fallback method + # for lc normalization when the call no does not conform to lc format. + # I'm unsure whether Sierra uses that as the fallback or if it falls back + # to char by char normalization.) + + module PhraseNormalization + # view index codes/names + # select * from sierra_view.phrase_type t + # inner join sierra_view.phrase_type_name n on n.phrase_type_id = t.id + # where n.iii_language_id = '1' + # order by varfield_type_code + # + # things where search is somewhat in place + # o OCLC# - works for oclc# in sierra + # i isbn/issn + # b barcode - works for barcodes in Sierra + # c lc call - works for lc call numbers. "not really lc call numbers" + # that are in an lc call number field generally work; + # there are ~1000 at UNC that we don't normalize correctly + # e local/dewey call - there are ~3 we don't normalize correctly + # g sudoc call - works for sudocs in Sierra + + # things we may want to search + # a t s d j (wxy)? + + # things where search is not in place + # k, q, a, t, h, s, d, r, f, j, p, l, n, z, 'v', 'u', 'y' + + def normalize(phrase, index = nil) + PhraseNormalization.normalize(phrase, index&.to_sym) + end + + def self.normalize(phrase, index = nil) + normalize_by_index_type(index, phrase) + end + + def self.normalize_by_index_type(index, phrase) + case index.to_sym + when :i, :b + number_normalize(phrase) + when :o + bib_utility_normalize(phrase) + when :a, :t, :s, :d, :j, nil + standard_normalize(phrase) + when :n + name_normalize(phrase) + when :c + lc_normalize(phrase) + when :e + dewey_normalize(phrase) + when :g + sudoc_normalize(phrase) + end + end + + def self.remove_punct(str) + # we need the spaces around and; dupe spaces will be removed + str = str.gsub('&', ' and ') + + # selectively remove punctuation: + # keep: + # +%$#@ + # remove: + str.gsub!(/["']/, '') + # replace with spaces: + str.gsub!(/[!\&'()*,\-.\/:;<=>?\[\\\]^_`{|}~]/, ' ') + str.squeeze!(' ') + str.strip! + str + end + + def self.test_pad_numbers(str, char: ' ') + str.gsub!(/(?\b)(?[0-9,-]+)(? *)/) do + head = $LAST_MATCH_INFO[:head] + orig = $LAST_MATCH_INFO[:orig] + tail = $LAST_MATCH_INFO[:tail] + sum = '' + orig.split('-').each_with_index do |hfrag, index| + sum << ' ' if index.positive? + next if hfrag.empty? + sum += + if orig.split('-')[index - 1]&.scan(/^[0-9]*/)&.first&.length == 4 + if hfrag[/^[0-9]{2}([^0-9]|$)/] + (hfrag.delete(',')&.rjust(0, char)).to_s + else + (hfrag.delete(',')&.rjust(8, char)).to_s + end + else + (hfrag.delete(',')&.rjust(8, char)).to_s + end + sum << ' ' if orig.end_with?('-') + end + sum << ' ' if orig == '-' + + head + sum + tail + end + str + end + + def self.bib_utility_normalize(str) + # Takes the first 150 characters of the string to be indexed + str = str[0..149] + + ## downcase str + str = str.downcase + + # Strips non-filing characters from titles as designated by MARC tag indicators + # ignore here + + # TODO: Remove select punctuation? + # We previously used this + # str.gsub!(/\u02B9|\u02BB|\uFE20|\uFE21/, '') # remove select punct + # before transliteration to remove special punct or diacritics + + # Strips apostrophes and diacritics + str.delete!('"\'') + str.gsub!(/[{}]/, ' ') + str = I18n.transliterate(str, replacement: '') + # Downcase anything transliterated into uppercase + str.downcase! + + # "Converts ampersands to the word for "and" in the primary language + # of your system" + # (we need the spaces around and; dupe spaces will be removed) + str.gsub!('&', ' and ') + + # Replace tildes with spaces. We'll replace other punct later, but + # this allows us to pad numbers with tildes and swap those tildes for + # spaces later + str.tr!('~', ' ') + + # number padding is affected by commas and hyphens; those chars still + # need to be present when we pad numbers. + punct_remove ||= /[.!\&'()*\/:;<=>?\[\\\]^_`]/ + str.gsub!(punct_remove, ' ') + str = test_pad_numbers(str, char: '~') + + str.gsub!(/\|./, ' ') + str.gsub!(/[\-,]/, ' ') + + # Collapses multiple spaces to a single space + str.squeeze!(' ') + + ### replace number padding with spaces + str.strip! + str.tr!('~', ' ') + + return nil if str == '' + str[0..124].rstrip + end + + def self.pad_numbers(str, char: ' ', pad_decimals: true, pad_length: 8) + # remove commas from comma-separated number groups + str = str.gsub(/(?<=[0-9]),(?=[0-9]{3})/, '') + + except_preceded_by = '\+#${' + except_preceded_by += '\.' unless pad_decimals + regexp = / + (??\-\[\\\]^_`|]/ + str.gsub!(punct_remove, ' ') + + # Collapses multiple spaces to a single space + str.squeeze!(' ') + + ### replace number padding with spaces + str.strip! + str.tr!('~', ' ') + + return nil if str == '' + + # This truncation does not work for some strings, seemingly with many + # multi-byte characters, e.g. long Russian titles. Those strings end up + # stored in Sierra's phrase_entry more heavily truncated than we truncate. + # e.g. b4954879a has a 490: + # "Anadolu'da Türk vatanı (1071, Malazgirt) ve Türk devleti (1075, İznik)nin kuruluşu 900. yıl dönümü hatırasına armaǧan ;" + # We normalize that to 123 chars (it's short enough we don't truncate): + # "anadoluda turk vatani 1071 malazgirt ve turk devleti 1075 iznik nin kurulusu 900 yil donumu hatirasina armagan" + # Sierra phrase_entry has (103 chars): + # "anadoluda turk vatani 1071 malazgirt ve turk devleti 1075 iznik nin kurulusu 900 yil donum" + # e.g. b28704939a has a 245: + # Zhurnal iskhod︠i︡ashchim bumagam kan︠t︡sel︠i︡arīi Moskovskago general-gubernatora grafa Rostopchina s ī︠i︡un︠i︡a po dekabrʹ 1812 goda" + # We normalize and truncate it to max 125 chars: + # "zhurnal iskhodiashchim bumagam kantseliarii moskovskago general gubernatora grafa rostopchina s iiunia po dekabr 1812 god" + # Sierra phrase_entry has (95 chars): + # "zhurnal iskhodiashchim bumagam kantseliarii moskovskago general gubernatora grafa rostopchina s" + # + # So, this could possibly be better. However, searching in the Sierra + # client for a longer title than is indexed also ends up returning no + # exact match results, so the problem is at least not unique to us. + str[0..124].rstrip + end + + def self.number_normalize(str) + str = str.downcase + str.gsub!(/["']/, '') + str.gsub!(/[$!\&'()*,.\/:;<=>\-?\[\\\]^_`{|}~]/, ' ') + str.delete!(' ') + str + end + + def self.name_normalize(str) + str = str.downcase + str.delete!('"\'') + str.gsub!(/[!\&'()*,.\/:;<=>\-?\[\\\]^_`{|}~]/, ' ') + str + end + + def self.lc_normalize(str) + str = str.dup + + vol_designators = 'no|v|sv|zv|liv|rev' + str.gsub!(/[{}]/, ' ') + str = I18n.transliterate(str, replacement: '') + # remove any "prestamp" before the call number starts + # 'a PS10.A1' => 'PS10.A1' + + regexp = / # 'blah PS3545.5 A1 1960' + ^(?.*?) # 'blah ' + (?(?[0-9]{1,4}) # '3545' + (?\.[0-9]+)? # '.5' + (?.*)? # ' A1 1960' + /x + + m = str.match(regexp) + return standard_normalize(str, pad_length: 0, punct_remove: /[!\&'()*,#\/:;<=>?\-\[\\\]^_`{|}]/) unless m + cls = m[:cls].downcase.ljust(3, ' ') + num = m[:num].rjust(4, ' ') + dec = m[:dec] + remainder = m[:remainder].downcase + return standard_normalize(str, pad_length: 0) if remainder =~ /^[0-9]/ + remainder.tr!('~', ' ') + remainder.gsub!(/(?<=[0-9])\.(?=[0-9])/, ' ') + remainder.gsub!(/(?\-?\[\\\]^_`|]/, ' ') + remainder.gsub!(/(?<=#{vol_designators})\.(?=[ 0-9])/, '~') + remainder.gsub!(/\./, '') + remainder = " #{remainder}" + + remainder.squeeze!(' ') + remainder.gsub!(/(?\b(#{vol_designators}))~ ?(?[0-9]+)/) do + caption = $LAST_MATCH_INFO[:cap] + numer = $LAST_MATCH_INFO[:numer] + caption + ' ' + numer.rjust(4, '~') + end + + remainder.gsub!(/([0-9])([a-z])/, '\1 \2') + remainder.squeeze!(' ') + remainder.gsub!(/~ /, ' ') + remainder.tr!('~', ' ') + + "#{cls}#{num}#{dec}#{remainder}".strip + end + + def self.dewey_normalize(str) + str.gsub!(/[{}]/, ' ') + str = I18n.transliterate(str, replacement: '') + str = str.downcase + str.gsub!(/(?\-?\[\\\]^_`|~]/, ' ') + str.gsub!('&', ' and ') + str.squeeze!(' ') + str.strip! + str = dewey_pad_numbers(str) + str.delete!(',') + str.rstrip + end + + def self.dewey_pad_numbers(str, char: ' ') + str = str.dup + str.gsub!(/(?\b|\.)(?[0-9,-.]+)(? *)/) do + head = $LAST_MATCH_INFO[:head] + orig = $LAST_MATCH_INFO[:orig] + tail = $LAST_MATCH_INFO[:tail] + pad = + orig.split('-').map { |hfrag| + csplit = hfrag.split(',') + sum = '' + csplit.each_with_index do |cfrag, index| + sum << ' ' if index.positive? + padding = + case csplit[index + 1]&.scan(/^[0-9]*/)&.first&.length + when 3 + 5 + when 4 + 4 + else + 8 + end + sum += (cfrag[/^[^.]+/]&.rjust(padding, char)).to_s + sum += cfrag[/\..*/].to_s + end + sum << ' ' if hfrag.end_with?(',') + sum + }.flatten.join(' ') + if !orig.match(/[0-9]/) || head == '.' + head + orig + tail + elsif pad.end_with?(' ') + pad + else + pad + tail + end + end + str + end + + def self.sudoc_normalize(str) + str.lstrip! + str.gsub!(/[{}]/, ' ') + str = I18n.transliterate(str, replacement: '') + str = str.downcase + str.squeeze!(' ') + str.gsub!(/ ?([a-z]) ?/, '\1') + + # remove spaces before/after select punctuation + # this is probably not a complete list of punctuation + # but it is likely a list of all puncuation in UNC's Sierra sudocs + str.gsub!(/ ?([#?\[\]&.()<>\/,;-]) ?/, '\1') + + str.gsub!(/ ?: ?/, ' :') + str.gsub!(/([0-9]+)/) { $1.rjust(5, ' ') } + str.rstrip! + str + end + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/helpers/sierra_marc.rb b/lib/sierra_postgres_utilities/data/helpers/sierra_marc.rb new file mode 100644 index 0000000..18731b0 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/helpers/sierra_marc.rb @@ -0,0 +1,78 @@ +require 'marc' +module Sierra + module Data + module Helpers + module SierraMARC + # Retrieves cached marc or compiles and caches marc + # + # @return [MARC::Record] + def marc + @marc ||= compile_marc + end + + # Compiles and caches marc, even if cached marc exists. + # + # @return [MARC::Record] + def compile_marc + @marc = Sierra::Data::Helpers::SierraMARC.compile_marc(self) + end + + # Compiles and caches marc, abnormally. + # + # This uses prepared statements which may be quicker than #marc / + # #compile_marc's use of associations for leader/control/varfields. + # However this method does not cache the leader/control/varfield values, + # it retrieves, so is likely slower if you also will need to retrieve + # those associations/fields for other reasons. + def quick_marc + @marc = Sierra::Data::Helpers::SierraMARC.compile_marc( + self, + ldr: @leader_field || Sierra::Data::LeaderField. + by_record_id(record_id: id), + cfs: @control_fields || Sierra::Data::ControlField. + by_record_id(record_id: id), + vfs: @varfields || Sierra::Data::Varfield. + by_record_id(record_id: id) + ) + end + + # Sets marc + # + # @param [MARC::Record] marc + # @return [MARC::Record] + def marc=(marc) + @marc = marc + end + + # Compiles marc for a record. + # + # Uses the record's leader/controlfield/varfield associations, unless + # passed leader/controlfield/varfield values. + # + # @param [Sierra::Data::Bib, Sierra::Data::Authority, ...] rec + # @param [Sierra::Data::LeaderField] ldr + # @param [Enumerable] cfs + # @param [Enumerable] vfs + # @return [MARC::Record] + def self.compile_marc(rec, ldr: nil, cfs: nil, vfs: nil) + ldr ||= rec.leader_field + cfs ||= rec.control_fields + vfs ||= rec.varfields + + m = MARC::Record.new + m.leader = ldr.to_s if ldr + + vfs.select(&:control_field?).map(&:to_marc).each do |cf| + m << cf if cf + end + cfs.each { |c| m << c.to_marc } + vfs.reject(&:control_field?).map(&:to_marc).each do |df| + m << df if df + end + + m + end + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/misc.rb b/lib/sierra_postgres_utilities/data/misc.rb new file mode 100644 index 0000000..a38461b --- /dev/null +++ b/lib/sierra_postgres_utilities/data/misc.rb @@ -0,0 +1,8 @@ +module Sierra + module Data + require_relative 'misc/phrase_entry.rb' + require_relative 'misc/checkout.rb' + require_relative 'misc/circ_trans.rb' + require_relative 'misc/create_list.rb' + end +end diff --git a/lib/sierra_postgres_utilities/data/misc/checkout.rb b/lib/sierra_postgres_utilities/data/misc/checkout.rb new file mode 100644 index 0000000..71e03a8 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/misc/checkout.rb @@ -0,0 +1,10 @@ +module Sierra + module Data + class Checkout < Sequel::Model(DB.db[:checkout]) + set_primary_key :id + + one_to_one :item, key: :id, primary_key: :item_record_id + one_to_one :patron, key: :id, primary_key: :patron_record_id + end + end +end diff --git a/lib/sierra_postgres_utilities/data/misc/circ_trans.rb b/lib/sierra_postgres_utilities/data/misc/circ_trans.rb new file mode 100644 index 0000000..6c1122e --- /dev/null +++ b/lib/sierra_postgres_utilities/data/misc/circ_trans.rb @@ -0,0 +1,12 @@ +module Sierra + module Data + class CircTrans < Sequel::Model(DB.db[:circ_trans]) + set_primary_key :id + + one_to_one :bib, key: :id, primary_key: :bib_record_id + one_to_one :item, key: :id, primary_key: :item_record_id + + one_to_one :patron, key: :id, primary_key: :patron_record_id + end + end +end diff --git a/lib/sierra_postgres_utilities/data/misc/create_list.rb b/lib/sierra_postgres_utilities/data/misc/create_list.rb new file mode 100644 index 0000000..bd7bb79 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/misc/create_list.rb @@ -0,0 +1,27 @@ +require_relative '../records/metadata.rb' + +module Sierra + module Data + class CreateList < Sequel::Model(DB.db[:bool_info]) + set_primary_key :id + + many_to_many :record_metadatas, + class: :'Sierra::Data::Metadata', left_key: :bool_info_id, + right_key: :record_metadata_id, join_table: :bool_set + + def records + record_metadatas.lazy.map(&:record) + end + + def empty? + count.zero? + end + + def self.get(num) + first(id: num) + end + end + + BoolInfo = CreateList + end +end diff --git a/lib/sierra_postgres_utilities/data/misc/phrase_entry.rb b/lib/sierra_postgres_utilities/data/misc/phrase_entry.rb new file mode 100644 index 0000000..2c8c763 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/misc/phrase_entry.rb @@ -0,0 +1,24 @@ +module Sierra + module Data + class PhraseEntry < Sequel::Model(DB.db[:phrase_entry]) + set_primary_key :id + + many_to_one :record_metadata, + class: :'Sierra::Data::Metadata', primary_key: :id, + key: :record_id + + many_to_one :bib, primary_key: :id, key: :record_id + many_to_one :item, primary_key: :id, key: :record_id + many_to_one :authority, primary_key: :id, key: :record_id + many_to_one :holdings, + class: :'Sierra::Data::Holdings', primary_key: :id, + key: :record_id + many_to_one :order, primary_key: :id, key: :record_id + many_to_one :patron, primary_key: :id, key: :record_id + + def record + record_metadata.record + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties.rb b/lib/sierra_postgres_utilities/data/properties.rb new file mode 100644 index 0000000..a8cbdaa --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties.rb @@ -0,0 +1,18 @@ +module Sierra + module Data + require_relative 'properties/bib_record_property' + require_relative 'properties/fund' + require_relative 'properties/holdings_card' + require_relative 'properties/item_record_property' + require_relative 'properties/item_status' + require_relative 'properties/itype' + require_relative 'properties/location' + require_relative 'properties/order_cmf' + require_relative 'properties/patron_address' + require_relative 'properties/patron_fullname' + require_relative 'properties/patron_phone' + require_relative 'properties/permission' + require_relative 'properties/ptype' + require_relative 'properties/varfield_type' + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/bib_record_property.rb b/lib/sierra_postgres_utilities/data/properties/bib_record_property.rb new file mode 100644 index 0000000..e5faed5 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/bib_record_property.rb @@ -0,0 +1,9 @@ +module Sierra + module Data + class BibRecordProperty < Sequel::Model(DB.db[:bib_record_property]) + set_primary_key :id + + one_to_one :bib, key: :id, primary_key: :bib_record_id + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/fund.rb b/lib/sierra_postgres_utilities/data/properties/fund.rb new file mode 100644 index 0000000..4ce488b --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/fund.rb @@ -0,0 +1,17 @@ +module Sierra + module Data + class Fund < Sequel::Model(DB.db[:fund_master]) + set_primary_key :code_num + + def cmfs + Sierra::Data::OrderCMF.where(fund_code: code_num.to_s.rjust(5, '0')). + all.lazy. + select { |cmf| cmf.accounting_unit_id == accounting_unit_id } + end + + def orders + cmfs.map(&:order) + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/holdings_card.rb b/lib/sierra_postgres_utilities/data/properties/holdings_card.rb new file mode 100644 index 0000000..5525760 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/holdings_card.rb @@ -0,0 +1,11 @@ +module Sierra + module Data + class HoldingsCard < Sequel::Model(DB.db[:holding_record_card]) + set_primary_key :id + + many_to_one :holdings, + class: :'Sierra::Data::Holdings', primary_key: :id, + key: :holding_record_id + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/item_record_property.rb b/lib/sierra_postgres_utilities/data/properties/item_record_property.rb new file mode 100644 index 0000000..c8ca64d --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/item_record_property.rb @@ -0,0 +1,9 @@ +module Sierra + module Data + class ItemRecordProperty < Sequel::Model(DB.db[:item_record_property]) + set_primary_key :id + + one_to_one :item, key: :id, primary_key: :item_record_id + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/item_status.rb b/lib/sierra_postgres_utilities/data/properties/item_status.rb new file mode 100644 index 0000000..c0d178b --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/item_status.rb @@ -0,0 +1,22 @@ +module Sierra + module Data + class ItemStatus < Sequel::Model(DB.db[:item_status_property]) + set_primary_key :id + + one_to_many :items, key: :item_status_code, primary_key: :code + + def name + @name ||= DB.db[:item_status_property_name]. + first(item_status_property_id: @values[:id])[:name] + end + + def self.list + order(:code).to_a.map { |x| [x.code, x.name] }.to_h + end + + def self.get(code) + first(code: code) + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/itype.rb b/lib/sierra_postgres_utilities/data/properties/itype.rb new file mode 100644 index 0000000..cb7a9ee --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/itype.rb @@ -0,0 +1,22 @@ +module Sierra + module Data + class Itype < Sequel::Model(DB.db[:itype_property]) + set_primary_key :id + + one_to_many :items, key: :itype_code_num, primary_key: :code_num + + def name + @name ||= DB.db[:itype_property_name]. + first(itype_property_id: @values[:id])[:name] + end + + def self.list + order(:code_num).to_a.map { |x| [x.code_num, x.name] }.to_h + end + + def self.get(code) + first(code_num: code) + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/location.rb b/lib/sierra_postgres_utilities/data/properties/location.rb new file mode 100644 index 0000000..5c799ee --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/location.rb @@ -0,0 +1,25 @@ +module Sierra + module Data + class Location < Sequel::Model(DB.db[:location]) + set_primary_key :id + + one_to_many :items, key: :location_code, primary_key: :code + many_to_many :bibs, + left_primary_key: :code, left_key: :location_code, + right_key: :bib_record_id, right_primary_key: :id, + join_table: :bib_record_location + + def name + @name ||= DB.db[:location_name].first(location_id: @values[:id])[:name] + end + + def self.list + order(:code).to_a.map { |x| [x.code, x.name] }.to_h + end + + def self.get(code) + first(code: code) + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/order_cmf.rb b/lib/sierra_postgres_utilities/data/properties/order_cmf.rb new file mode 100644 index 0000000..fdd45c5 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/order_cmf.rb @@ -0,0 +1,20 @@ +module Sierra + module Data + class OrderCMF < Sequel::Model(DB.db[:order_record_cmf]) + set_primary_key :id + + many_to_one :order, primary_key: :record_id, key: :order_record_id + one_to_one :location, key: :code, primary_key: :location_code + + def fund + Sierra::Data::Fund.first(code_num: fund_code, + accounting_unit_id: accounting_unit_id) + end + + def accounting_unit_id + DB.db[:accounting_unit]. + first(code_num: order.accounting_unit_code_num)[:id] + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/patron_address.rb b/lib/sierra_postgres_utilities/data/properties/patron_address.rb new file mode 100644 index 0000000..f647e11 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/patron_address.rb @@ -0,0 +1,9 @@ +module Sierra + module Data + class PatronAddress < Sequel::Model(DB.db[:patron_record_address]) + set_primary_key :id + + many_to_one :patron, primary_key: :id, key: :patron_record_id + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/patron_fullname.rb b/lib/sierra_postgres_utilities/data/properties/patron_fullname.rb new file mode 100644 index 0000000..ee94a49 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/patron_fullname.rb @@ -0,0 +1,23 @@ +module Sierra + module Data + class PatronFullname < Sequel::Model(DB.db[:patron_record_fullname]) + set_primary_key :id + + many_to_one :patron, primary_key: :id, key: :patron_record_id + + def full + [first_name, middle_name, last_name, suffix]. + join(' '). + gsub(/\s+/, ' '). + strip + end + + def full_reverse + [last_name, first_name, middle_name, suffix]. + join(' '). + gsub(/\s+/, ' '). + strip + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/patron_phone.rb b/lib/sierra_postgres_utilities/data/properties/patron_phone.rb new file mode 100644 index 0000000..329c2dd --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/patron_phone.rb @@ -0,0 +1,9 @@ +module Sierra + module Data + class PatronPhone < Sequel::Model(DB.db[:patron_record_phone]) + set_primary_key :id + + many_to_one :patron, primary_key: :id, key: :patron_record_id + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/permission.rb b/lib/sierra_postgres_utilities/data/properties/permission.rb new file mode 100644 index 0000000..88ff72c --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/permission.rb @@ -0,0 +1,30 @@ +module Sierra + module Data + class Permission < Sequel::Model(DB.db[:iii_user_permission_myuser]) + set_primary_key :id + + one_to_many :user, primary_key: :iii_user_id, key: :id + many_to_many :users, + left_key: :permission_num, left_primary_key: :permission_num, + right_key: :iii_user_id, right_primary_key: :id, + join_table: :iii_user_permission_myuser + + def record + @record ||= bib_record || item_record + end + + # get a list of permissions (not a list of all user-permissions) + def self.list + distinct. + order(:permission_num). + to_a. + map { |x| [x.permission_num, x.permission_name] }. + to_h + end + + def self.get(num) + first(permission_num: num) + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/ptype.rb b/lib/sierra_postgres_utilities/data/properties/ptype.rb new file mode 100644 index 0000000..94f621d --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/ptype.rb @@ -0,0 +1,24 @@ +module Sierra + module Data + class Ptype < Sequel::Model(DB.db[:ptype_property]) + set_primary_key :id + + one_to_many :patrons, key: :ptype_code_num, primary_key: :code_num + + alias code_num id + + def name + @name ||= DB.db[:ptype_property_name]. + first(ptype_id: @values[:id])[:description] + end + + def self.list + order(:id).to_a.map { |x| [x.id, x.name] }.to_h + end + + def self.get(ptype) + first(id: ptype) + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/properties/varfield_type.rb b/lib/sierra_postgres_utilities/data/properties/varfield_type.rb new file mode 100644 index 0000000..91f7017 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/properties/varfield_type.rb @@ -0,0 +1,26 @@ +module Sierra + module Data + class VarfieldType < Sequel::Model(DB.db[:varfield_type]) + set_primary_key :id + + # TODO: associate? + + def name + @name ||= DB.db[:varfield_type_name]. + first(varfield_type_id: @values[:id])[:name] + end + + def short_name + @short_name ||= DB.db[:varfield_type_name]. + first(varfield_type_id: @values[:id])[:short_name] + end + + def self.list(record_type_code) + where(record_type_code: record_type_code). + order(:code).to_a. + map { |x| [x.code, x.name] }. + to_h + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/records.rb b/lib/sierra_postgres_utilities/data/records.rb new file mode 100644 index 0000000..e2d3b75 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records.rb @@ -0,0 +1,14 @@ +module Sierra + module Data + require_relative 'records/deleted' + require_relative 'records/metadata' + + require_relative 'records/generic' + require_relative 'records/authority' + require_relative 'records/bib' + require_relative 'records/holdings' + require_relative 'records/item' + require_relative 'records/order' + require_relative 'records/patron' + end +end diff --git a/lib/sierra_postgres_utilities/data/records/authority.rb b/lib/sierra_postgres_utilities/data/records/authority.rb new file mode 100644 index 0000000..6604a8b --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records/authority.rb @@ -0,0 +1,29 @@ +module Sierra + module Data + class Authority < Sequel::Model(DB.db[:record_metadata]. + inner_join(:authority_record, [:id])) + include Sierra::Data::GenericRecord + include Sierra::Data::Helpers::SierraMARC + + set_primary_key :id + prepare_retrieval_by :record_num, :first + + # Common to records + one_to_one :record_metadata, + class: :'Sierra::Data::Metadata', key: :id + one_to_many :control_fields, + class: :'Sierra::Data::ControlField', key: :record_id, + order: %i[varfield_type_code occ_num id] + one_to_one :leader_field, + key: :record_id + one_to_many :varfields, + key: :record_id, + order: %i[marc_tag varfield_type_code occ_num id] + one_to_many :subfields, + key: :record_id + one_to_many :phrase_entries, + key: :record_id, + order: %i[index_tag varfield_type_code occurrence id] + end + end +end diff --git a/lib/sierra_postgres_utilities/data/records/bib.rb b/lib/sierra_postgres_utilities/data/records/bib.rb new file mode 100644 index 0000000..c090b1c --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records/bib.rb @@ -0,0 +1,151 @@ +module Sierra + module Data + class Bib < Sequel::Model(DB.db[:record_metadata]. + inner_join(:bib_record, [:id])) + include Sierra::Data::GenericRecord + include Sierra::Data::Helpers::SierraMARC + + attr_writer :stub + + set_primary_key :id + prepare_retrieval_by :record_num, :first + + # Common to records + one_to_one :record_metadata, + class: :'Sierra::Data::Metadata', key: :id + one_to_many :control_fields, + class: :'Sierra::Data::ControlField', key: :record_id, + order: %i[varfield_type_code occ_num id] + one_to_one :leader_field, + key: :record_id + one_to_many :varfields, + key: :record_id, + order: %i[marc_tag varfield_type_code occ_num id] + one_to_many :subfields, + key: :record_id + one_to_many :phrase_entries, + key: :record_id, + order: %i[index_tag varfield_type_code occurrence id] + + # Attributes/properties + many_to_many :locations, + left_key: :bib_record_id, right_key: :location_code, + right_primary_key: :code, join_table: :bib_record_location, + order: :location_code + one_to_one :property, + class: :'Sierra::Data::BibRecordProperty', + key: :bib_record_id, primary_key: :id + + # Attachments + many_to_many :items, + left_key: :bib_record_id, right_key: :item_record_id, + join_table: :bib_record_item_record_link, + order: :items_display_order + many_to_many :holdings, + class: :'Sierra::Data::Holdings', + left_key: :bib_record_id, right_key: :holding_record_id, + join_table: :bib_record_holding_record_link, + order: :holdings_display_order + many_to_many :orders, + left_key: :bib_record_id, right_key: :order_record_id, + join_table: :bib_record_order_record_link, + order: :orders_display_order + + # Other + one_to_many :circ_trans, + class: :'Sierra::Data::CircTrans', key: :bib_record_id + one_to_many :holds, + key: :record_id + + # Returns array of call number prefix(es) + # e.g. ["n", "pq"] + def call_number_prefixes + @call_number_prefixes ||= DB.db[:bib_record_call_number_prefix]. + where(bib_record_id: @values[:id]). + map { |r| r[:call_number_prefix] } + end + + ##### Logic + alias bnum rnum + alias bnum_trunc rnum_trunc + alias bnum_with_check rnum_with_check + + alias cat_date cataloging_date_gmt + + def mat_type + property[:material_code] + end + + # @return [Array] record's location code(s) excepting "multi" + def location_codes + locations.map(&:code).reject { |c| c == 'multi' } + end + + def best_title + property.best_title + end + + def best_author + property.best_author + end + + # Returns record's imprint. + # + # Uses the first 260/264 by occ_num. + # + # @return [String] record's imprint + # @example + # bib.marc['260'].to_mrk + # #=> "=260 \\\\$aSanta Barbara :$bBlack Sparrow Press,$c1976." + # bib.imprint + # #=> "Santa Barbara : Black Sparrow Press, 1976." + def imprint + varfields. + select { |v| v.marc_tag =~ /26[04]/ }. + min_by(&:occ_num). + field_content. + gsub(/\|./, ' ').lstrip + end + + # LDR/06 + def rec_type + leader_field&.record_type_code + end + + # LDR/07; bcode1 also represents blvl and is not always the same + def blvl + leader_field&.bib_level_code + end + + # LDR/08 + def ctrl_type + leader_field&.control_type_code + end + + # Record's OCLC# derived from MARC record. + # + # Uses UNC-specific logic + # @return [String] oclc number + def oclcnum + marc.oclcnum + end + + # Creates a marc stub record for batch loading. + # + # Stub contains: + # - a 907 with bnum suitable for overlaying + # - a 944 with batch load note template + # + # @return [MARC::Record] + def stub + return @stub if @stub + @stub = MARC::Record.new + @stub << MARC::DataField.new('907', ' ', ' ', ['a', ".#{bnum}"]) + load_note = + 'Batch load history: 999 Something records loaded 20190000, xxx.' + @stub << MARC::DataField.new('944', ' ', ' ', ['a', load_note]) + @stub + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/records/deleted.rb b/lib/sierra_postgres_utilities/data/records/deleted.rb new file mode 100644 index 0000000..6a32804 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records/deleted.rb @@ -0,0 +1,30 @@ +module Sierra + module Data + # Used for deleted records in place of Sierra::Data::Bib, ::Item, etc. + # Deleted records have access to data/methods available through + # record_metadata but lack data/methods from their record_type class. + module DeletedRecord + # Error to raise when treating a DeletedRecord as if it were not deleted. + class DeletedRecordError < StandardError + end + + def deleted? + true + end + + # Raise DeletedRecordError when method is missing. + # + # We don't know whether it was a bib/item/other_record_type method + # being called on this deleted record, but we want to warn especially + # in case it was. + def method_missing + raise DeletedRecordError, "Deleted record #{record_type_code}" \ + "#{record_num} lacks methods associated with undeleted records." + end + + def respond_to_missing?(*) + true + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/records/generic.rb b/lib/sierra_postgres_utilities/data/records/generic.rb new file mode 100644 index 0000000..d78589b --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records/generic.rb @@ -0,0 +1,132 @@ +module Sierra + module Data + # Methods common to Sierra records (Bib, Item, etc. -- things reflected in + # record_metadata.) + module GenericRecord + # @example + # #=> "#450972566081, ... + # :is_available_at_library=>true}>" + def inspect + "#<#{self.class} #{rnum} @values=#{values}>" + end + + # @example + # #=> "#" + def to_s + "#<#{self.class} #{rnum}>" + end + + ##################### + # @!group Record id/number formats + # record_id = 420907889860 + # rnum = 'b1094852a' + # rnum_trunc = 'b1094852' + # rnum_with_check = 'b10948521' + # recnum = '1094852' + + # @return [Bigint] id / record_id (e.g. 420907889860) + def record_id + id + end + + # @return [String] rnum (e.g. 'b1094852a') + def rnum + @rnum ||= "#{record_type_code}#{record_num}a" + end + + # @return [String] rnum without check digit (e.g. 'b1094852') + def rnum_trunc + rnum.chop + end + + # @return [String] rnum with check digit (e.g. 'b10948521') + def rnum_with_check + rnum.chop + check_digit(recnum) + end + + # @return [String] record_num (e.g. '1094852') + def recnum + record_num.to_s + end + + # @!endgroup + + # @param [String] recnum record_num + # @return [String] check digit for given recnum + def check_digit(recnum) + digits = recnum.split('').reverse + y = 2 + sum = 0 + digits.each do |digit| + sum += digit.to_i * y + y += 1 + end + remainder = sum % 11 + if remainder == 10 + 'x' + else + remainder.to_s + end + end + + ##################### + + def suppressed? + is_suppressed + end + + def deleted? + return true if deletion_date_gmt + false + end + + # @return [String] record's record_type_code (e.g. "b", "i", etc.) + def type + record_type_code + end + + # @return [Time] record's creation date + def created_date + creation_date_gmt + end + + # @return [Time] record's updated date + def updated_date + record_last_updated_gmt + end + + ###################### + # MARC fields and non-MARC varfields + ####### + + def varfield_search(tag_or_type, value_only: true) + vfs = + if tag_or_type =~ /\d{3}/ + varfields.select { |v| v.marc_tag == tag_or_type } + else + varfields.select { |v| v.varfield_type_code == tag_or_type } + end + return vfs.map(&:field_content) if value_only + vfs + end + + # @return [Hash] mapping of vf codes to names for record's + # type + # @example item varfield code=>types + # # Sierra::Data::Item.first.vf_codes + # #=> {..., "a"=>"DRA Item Field", "b"=>"Barcode", "c"=>"Call No.", + # "d"=>"DRA Created Date", "f"=>"Library", ... } + def vf_codes + Sierra::Data::VarfieldType.list(type) + end + + #### + + # @param [String, Int] list_num review_file / list number + # @return [Boolean] whether record is present in specified list + def in_list?(list_num) + create_lists.any? { |l| l.id == list_num.to_i } + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/records/holdings.rb b/lib/sierra_postgres_utilities/data/records/holdings.rb new file mode 100644 index 0000000..693978e --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records/holdings.rb @@ -0,0 +1,56 @@ +module Sierra + module Data + class Holdings < Sequel::Model(DB.db[:record_metadata]. + inner_join(:holding_record, [:id])) + include Sierra::Data::GenericRecord + include Sierra::Data::Helpers::SierraMARC + + set_primary_key :id + + # Common to records + one_to_one :record_metadata, + class: :'Sierra::Data::Metadata', key: :id + one_to_many :control_fields, + class: :'Sierra::Data::ControlField', key: :record_id, + order: %i[varfield_type_code occ_num id] + one_to_one :leader_field, + key: :record_id + one_to_many :varfields, + key: :record_id, + order: %i[marc_tag varfield_type_code occ_num id] + one_to_many :subfields, + key: :record_id + one_to_many :phrase_entries, + key: :record_id, + order: %i[index_tag varfield_type_code occurrence id] + many_to_many :create_lists, + class: :'Sierra::Data::CreateList', right_key: :bool_info_id, + left_key: :record_metadata_id, left_primary_key: :id, + join_table: :bool_set + + # Attributes/properties + one_to_many :cards, + class: :'Sierra::Data::HoldingsCard', key: :holding_record_id, + order: :id + many_to_many :locations, + left_key: :holding_record_id, right_key: :location_code, + right_primary_key: :code, + join_table: :holding_record_location, order: :display_order + + # Attachments + one_through_one :bib, + left_key: :holding_record_id, right_key: :bib_record_id, + join_table: :bib_record_holding_record_link + many_to_many :items, + left_key: :holding_record_id, right_key: :item_record_id, + join_table: :holding_record_item_record_link, + order: :items_display_order + + #### Logic + + def card_count + cards.length + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/records/item.rb b/lib/sierra_postgres_utilities/data/records/item.rb new file mode 100644 index 0000000..fe0d4b6 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records/item.rb @@ -0,0 +1,184 @@ +module Sierra + module Data + # Model for item records. + # + # For items we cannot use a natural join between record_metadata and + # item_record. The rm.agency_code_num only ever seems to be 0 and that + # is not the case for i.agency_code_num. This should only be a problem + # for item records. + # We inner join using "USING id" (i.e. "[:id]") syntax to avoid duplicate + # id columns and potential AmbiguousColumn errors. + class Item < Sequel::Model(DB.db[:record_metadata]. + inner_join(:item_record, [:id])) + include Sierra::Data::GenericRecord + + set_primary_key :id + prepare_retrieval_by :record_num, :first + + # Common to records + one_to_one :record_metadata, + class: :'Sierra::Data::Metadata', key: :id + one_to_many :control_fields, + class: :'Sierra::Data::ControlField', key: :record_id, + order: %i[varfield_type_code occ_num id] + one_to_one :leader_field, + key: :record_id + one_to_many :varfields, + key: :record_id, + order: %i[marc_tag varfield_type_code occ_num id] + one_to_many :subfields, + key: :record_id + one_to_many :phrase_entries, + key: :record_id, + order: %i[index_tag varfield_type_code occurrence id] + + # Attributes/properties + many_to_one :location, primary_key: :code, key: :location_code + many_to_one :itype_property, + class: :'Sierra::Data::Itype', primary_key: :code_num, + key: :itype_code_num + many_to_one :item_status_property, + class: :'Sierra::Data::ItemStatus', primary_key: :code, + key: :item_status_code + one_to_one :property, + class: :'Sierra::Data::ItemRecordProperty', + key: :item_record_id, primary_key: :id + + # Attachments + many_to_many :bibs, + left_key: :item_record_id, right_key: :bib_record_id, + join_table: :bib_record_item_record_link + one_through_one :holdings, + left_key: :item_record_id, right_key: :holding_record_id, + join_table: :holding_record_item_record_link + + # Other + one_to_many :circ_trans, + class: :'Sierra::Data::CircTrans', key: :item_record_id + one_to_many :holds, key: :record_id + one_to_one :checkout, key: :item_record_id + + alias itype itype_property + alias status item_status_property + + #### Logic + + alias inum rnum + alias inum_trunc rnum_trunc + alias inum_with_check rnum_with_check + + # @param [Boolean] value_only (default: true) whether to return data as + # strings of the field value/content? When false, returns data as entire + # Sierra::Data::Varfields + # @return [Array] record's barcode(s) data + def barcodes(value_only: true) + varfield_search('b'.freeze, value_only: value_only) + end + + # @param (see #barcodes) + # @return [Array] record's "Library" varfield data + def varfield_librarys(value_only: true) + varfield_search('f'.freeze, value_only: value_only) + end + + # @param (see #barcodes) + # @return [Array] record's "Stats" varfield data + def stats_fields(value_only: true) + varfield_search('j'.freeze, value_only: value_only) + end + + # @param (see #barcodes) + # @return [Array] record's message field data + def messages(value_only: true) + varfield_search('m'.freeze, value_only: value_only) + end + + # @param (see #barcodes) + # @return [Array] record's volume field data + def volumes(value_only: true) + varfield_search('v'.freeze, value_only: value_only) + end + + # @param (see #barcodes) + # @return [Array] record's internal_notes data + def internal_notes(value_only: true) + varfield_search('x'.freeze, value_only: value_only) + end + + # @param (see #barcodes) + # @return [Array] record's public_notes data + def public_notes(value_only: true) + varfield_search('z'.freeze, value_only: value_only) + end + + # Returns record's call number data. + # + # Subfield delimiters are stripped unless keep_delimiters: true + # + # @example data without delimiters + # item.callnos + # #=> ["PR6056.A82 S6"] + # + # @example data with delimiters + # item.callnos(keep_delimiters: true) + # #=> ["|aPR6056.A82 S6"] + # + # @example data as whole Varfields + # item.callnos(value_only: false) + # #=> [#114483, + # :record_id=>450972566081, ..., :field_content=>"|aPR6056.A82 S6" + # }>] + # + # @param [Boolean] keep_delimiters (default: false) retain subfield + # delimiters when returning data as strings? + # @param [Boolean] value_only (default: true) whether to return data as + # strings of the field value/content? When false, returns data as entire + # Sierra::Data::Varfields + # @return [Array] record's public_notes data + def callnos(value_only: true, keep_delimiters: false) + cns = varfield_search('c'.freeze, value_only: value_only) + if value_only && !keep_delimiters + cns&.map { |x| x.gsub(/\|./, '').strip } + else + cns + end + end + + def checked_out? + !checkout.nil? + end + + # @return [Time, nil] due_date if item is checked out + def due_date + return unless checked_out? + + checkout.due_gmt + end + + # @todo deprecate? is this used somewhere external? + def itype_code + itype_code_num.to_s + end + + # @return [String] itype description/name (e.g. "Book") + def itype_desc + itype.name + end + + # @return [String] location description/name (e.g. "Davis Library") + def location_desc + location.name + end + + # @return [String] item_status_code (e.g. "-") + def status_code + item_status_code + end + + # @return [String] item status description/name (e.g. "Available") + def status_desc + status.name.capitalize + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/records/metadata.rb b/lib/sierra_postgres_utilities/data/records/metadata.rb new file mode 100644 index 0000000..4563538 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records/metadata.rb @@ -0,0 +1,58 @@ +module Sierra + module Data + class Metadata < Sequel::Model(DB.db[:record_metadata]) + set_primary_key :id + prepare_retrieval_by :id, :first + + # Common to records + one_to_many :control_fields, + class: :'Sierra::Data::ControlField', key: :record_id, + order: %i[varfield_type_code occ_num id] + one_to_one :leader_field, + key: :record_id + one_to_many :varfields, + key: :record_id, + order: %i[marc_tag varfield_type_code occ_num id] + one_to_many :subfields, + key: :record_id + one_to_many :phrase_entries, + key: :record_id, + order: %i[index_tag varfield_type_code occurrence id] + + one_to_one :bib, key: :record_id + one_to_one :item, key: :record_id + one_to_one :authority, key: :record_id + one_to_one :holdings, + class: :'Sierra::Data::Holdings', key: :record_id + one_to_one :order, key: :record_id + one_to_one :patron, key: :record_id + + def record + if deletion_date_gmt + extend Sierra::Data::DeletedRecord + return @record = self + end + @record ||= + case record_type_code + when 'b' + bib + when 'i' + item + when 'c' + holdings + when 'o' + order + when 'a' + authority + when 'p' + patron + end + end + + def deleted? + return true if deletion_date_gmt + false + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/records/order.rb b/lib/sierra_postgres_utilities/data/records/order.rb new file mode 100644 index 0000000..2c89889 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records/order.rb @@ -0,0 +1,56 @@ +module Sierra + module Data + class Order < Sequel::Model(DB.db[:record_metadata]. + inner_join(:order_record, [:id])) + include Sierra::Data::GenericRecord + + set_primary_key :id + prepare_retrieval_by :record_num, :first + + # Common to records + one_to_one :record_metadata, + class: :'Sierra::Data::Metadata', key: :id + one_to_many :control_fields, + class: :'Sierra::Data::ControlField', key: :record_id, + order: %i[varfield_type_code occ_num id] + one_to_one :leader_field, + key: :record_id + one_to_many :varfields, + key: :record_id, + order: %i[marc_tag varfield_type_code occ_num id] + one_to_many :subfields, + key: :record_id + one_to_many :phrase_entries, + key: :record_id, + order: %i[index_tag varfield_type_code occurrence id] + + # Attributes/properties + one_to_many :cmfs, + class: :'Sierra::Data::OrderCMF', key: :order_record_id, + order: :display_order + + # Attachments + one_through_one :bib, + left_key: :order_record_id, right_key: :bib_record_id, + join_table: :bib_record_order_record_link + + alias cat_date catalog_date_gmt + alias received_date received_date_gmt + alias status_code order_status_code + + def funds + @funds ||= cmfs.map(&:fund).uniq + end + + #### Logic + + def number_copies + cmfs.map(&:copies) + end + + def location + cmfs.map(&:location_code) + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/records/patron.rb b/lib/sierra_postgres_utilities/data/records/patron.rb new file mode 100644 index 0000000..b50f368 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/records/patron.rb @@ -0,0 +1,75 @@ +module Sierra + module Data + class Patron < Sequel::Model(DB.db[:record_metadata]. + inner_join(:patron_record, [:id])) + include Sierra::Data::GenericRecord + + set_primary_key :id + prepare_retrieval_by :record_num, :first + + # Common to records + one_to_one :record_metadata, + class: :'Sierra::Data::Metadata', key: :id + one_to_many :control_fields, + class: :'Sierra::Data::ControlField', key: :record_id, + order: %i[varfield_type_code occ_num id] + one_to_one :leader_field, + key: :record_id + one_to_many :varfields, + key: :record_id, + order: %i[marc_tag varfield_type_code occ_num id] + one_to_many :subfields, + key: :record_id + one_to_many :phrase_entries, + key: :record_id, + order: %i[index_tag varfield_type_code occurrence id] + + # Attributes/properties + one_to_one :ptype_property, + class: :'Sierra::Data::Ptype', primary_key: :ptype_code, + key: :value + one_to_many :addresses, + class: :'Sierra::Data::PatronAddress', + key: :patron_record_id, order: :display_order + one_to_many :names, + class: :'Sierra::Data::PatronFullname', + key: :patron_record_id, order: :display_order + one_to_many :phones, + class: :'Sierra::Data::PatronPhone', key: :patron_record_id, + order: :display_order + + # Other + one_to_many :circ_trans, + class: :'Sierra::Data::CircTrans', key: :patron_record_id + one_to_many :holds, key: :patron_record_id + one_to_many :checkouts, key: :patron_record_id + + alias ptype ptype_property + alias expiration_date expiration_date_gmt + + #### Logic + + # Sierra manual: An expired patron is one with an EXP DATE fixed-length + # field value earlier than or equal to the current date + def expired? + expiration_date <= Time.now + end + + def barcodes(value_only: true) + varfield_search('b'.freeze, value_only: value_only) + end + + def emails(value_only: true) + varfield_search('z'.freeze, value_only: value_only) + end + + def name + names.first.full + end + + def name_reverse + names.first.full_reverse + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/secondary_records.rb b/lib/sierra_postgres_utilities/data/secondary_records.rb new file mode 100644 index 0000000..a23b4a8 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/secondary_records.rb @@ -0,0 +1,13 @@ +# It seems useful to draw a distinction between "records" which are reflected +# in record_metadata (i.e. bib, item, etc.) and record-like objects not +# present in record_metadata. They're being grouped here as "secondary records" +# or "second-class records." because they seem more record-like than, say, +# itype, but that distinction is possibly nebulous and of questionable +# usefulness. + +module Sierra + module Data + require_relative 'secondary_records/hold' + require_relative 'secondary_records/user' + end +end diff --git a/lib/sierra_postgres_utilities/data/secondary_records/hold.rb b/lib/sierra_postgres_utilities/data/secondary_records/hold.rb new file mode 100644 index 0000000..ac9abe5 --- /dev/null +++ b/lib/sierra_postgres_utilities/data/secondary_records/hold.rb @@ -0,0 +1,47 @@ +module Sierra + module Data + class Hold < Sequel::Model(DB.db[:hold]) + set_primary_key :id + + many_to_one :record_metadata, + class: :'Sierra::Data::Metadata', primary_key: :id, + key: :record_id + many_to_one :patron, primary_key: :id, key: :patron_record_id + + many_to_one :bib, primary_key: :id, key: :record_id + many_to_one :item, primary_key: :id, key: :record_id + + def record + record_metadata.record + end + + def type + Hold.type(record.type) + end + + def status_desc + Hold.status_desc(status) + end + + def self.type(record_type) + case record_type + when 'b' + 'bib' + when 'i' + 'item' + end + end + + def self.status_desc(status_code) + case status_code + when '0' + 'On hold.' + when 'i', 'b', 'j' + 'Ready for pickup.' + when 't' + 'In transit to pickup.' + end + end + end + end +end diff --git a/lib/sierra_postgres_utilities/data/secondary_records/user.rb b/lib/sierra_postgres_utilities/data/secondary_records/user.rb new file mode 100644 index 0000000..5b2228e --- /dev/null +++ b/lib/sierra_postgres_utilities/data/secondary_records/user.rb @@ -0,0 +1,26 @@ +module Sierra + module Data + class User < Sequel::Model(DB.db[:iii_user]) + set_primary_key :id + + one_to_many :permissions, + class: :'Sierra::Data::Permission', key: :iii_user_id, + primary_key: :id + + def record + @record ||= bib_record || item_record + end + + def pretty_permissions + puts permissions. + sort_by(&:permission_num). + map { |p| "#{p.permission_num}\t#{p.permission_name}" }. + join("\n") + end + + def self.get(login) + Sierra::Data::User.first(name: login) + end + end + end +end diff --git a/lib/sierra_postgres_utilities/db.rb b/lib/sierra_postgres_utilities/db.rb new file mode 100644 index 0000000..4460264 --- /dev/null +++ b/lib/sierra_postgres_utilities/db.rb @@ -0,0 +1,19 @@ +require 'sequel' + +Sequel::Model.plugin :dataset_associations + +module Sierra + # Includes Sierra database connection, direct querying, export functions. + module DB + require_relative 'db/connection' + require_relative 'db/query' + + extend Sierra::DB::Connection + extend Sierra::DB::Query + + # Connects using default credentials / connection options. + # Setting ENV['SIERRA_DELAY_CONNECT'] defers connection and allows + # connection using custom credentials / options. + Sierra::DB.connect unless ENV['SIERRA_DELAY_CONNECT'] + end +end diff --git a/lib/sierra_postgres_utilities/db/connection.rb b/lib/sierra_postgres_utilities/db/connection.rb new file mode 100644 index 0000000..62b0cc5 --- /dev/null +++ b/lib/sierra_postgres_utilities/db/connection.rb @@ -0,0 +1,132 @@ +require 'yaml' +require 'pg' +require 'sequel' + +module Sierra + module DB + module Connection + CREDS = {} + + # @return [Sequel::Database] the Sequel::Database "connection" to the db. + def db + Sierra::DB::Connection.db + end + + # Whether a valid connection to Sierra database exists. + # + # @return [Boolean] + def connected? + Sierra::DB::Connection.connected? + end + + # Establish connection to Sierra postgres database. + # + # @param [String, Hash] creds optionally specify credentials to use + # + # If given, creds can be a: + # - String containing the filepath to a yaml file containing creds + # - Hash containing the creds + # If creds is not given, connects using the first existing of these: + # - Sierra::DB::Connection::CREDS (Hash), if populated + # - SIERRA_INIT_CREDS (String) environment variable + # - 'sierra_prod.secret' + # yaml files are searched for in this order: + # - working directory + # - sierra_postgres_utilities install / base directory + # + # Credentials needed are: host, post, username, password. + # @param [Hash] options options to pass directly to Sequel.connect + # + # @see http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html#label-General+connection+options + def connect(creds: nil, options: {}) + return if connected? + Sierra::DB::Connection.connect(creds, options: options) + end + + # (see #db) + def self.db + @db + end + + # (see #connected?) + def self.connected? + @db.test_connection + rescue + false + end + + # (see #connect) + def self.connect(creds = nil, options: {}) + set_creds(creds) if creds || Sierra::DB::Connection::CREDS.empty? + make_connection(Sierra::DB::Connection::CREDS.merge(options)) + end + + # Returns a hash of credentials along with some connection constants. + # Accepts a hash or a yaml filename (expected to contain connection + # variables). Tries default credential locations if nothing given. + # @param [Hash, String] creds hash of credentials or string with + # path to yaml file containing credentials. see #connect + # @return [Hash, nil] hash of connection credentials and Sierra + # connection constants. nil if valid-looking credentials were + # not given/found. + def self.set_creds(creds) + creds ||= ENV['SIERRA_INIT_CREDS'] || 'sierra_prod.secret' + creds = creds_from_file(creds) unless creds.is_a?(Hash) + return unless creds + + CREDS['host'] = creds['host'] + CREDS['port'] = creds['port'] + CREDS['user'] = creds['user'] + CREDS['password'] = creds['password'] + + CREDS['database'] = 'iii' + CREDS['adapter'] = 'postgres' + CREDS['search_path'] = 'sierra_view' + end + + # Establishes connection to Sierra database + # + # If connection is established, tries to reload any + # sierra_postgres_utilities files that are skipped when a valid + # connection does not exist. + # + # @param [Hash] creds Here, creds is a hash containing + # - credentials (host/port/username/constant) + # - Sierra connection constants (database/adapter/search_path) + # - any other connection options previously supplied + # @return void + def self.make_connection(creds) + @db = Sequel::Database.connect(creds) + return unless connected? + + # If lack of a connection caused loading of some files to be skipped, + # go back and load them + load File.join(base_dir, 'lib/sierra_postgres_utilities.rb') + end + + # Reads credentials from yaml file. + # + # @param [String] path to credentials yaml file + # @return [Hash] credentials read from file + def self.creds_from_file(file) + begin + creds = YAML.load_file(file) + rescue Errno::ENOENT + begin + creds = YAML.load_file(File.join(base_dir, file)) + rescue Errno::ENOENT + puts 'WARN: Connection credentials invalid or not found.' + end + end + creds + end + + # @return [String] sierra_postgres_utilities base/install directory + # @example + # "#=> path/to/sierra-postgres-utilities" + def self.base_dir + File.dirname(File.expand_path('../..', __dir__)).to_s + end + end + end +end diff --git a/lib/sierra_postgres_utilities/db/query.rb b/lib/sierra_postgres_utilities/db/query.rb new file mode 100644 index 0000000..2bc8603 --- /dev/null +++ b/lib/sierra_postgres_utilities/db/query.rb @@ -0,0 +1,209 @@ +require 'csv' +require 'yaml' +require 'mail' + +module Sierra + module DB + # Manual/direct sql querying and exporting. + module Query + def query(query) + Sierra::DB::Query.make_query(query) + end + + def results + Sierra::DB::Query.results + end + + # Writes results to file. + # + # Formats: tsv, csv, xlsx (xlsx writable on windows only) + # + # @param [String] outfile path for outfile + # - for xlsx only: a relative path is relative to user's windows + # home directory, so using an absolute path may be preferable + # @param [Enumerable<#values>] results (default: Sierra::DB.results) + # @param [Boolean] include_headers (default: true) write headers to file? + # @param [Symbol] format (default: tsv) format of export: :tsv, :csv, + # :xlsx. + def write_results(outfile, results: self.results, + include_headers: true, format: :tsv) + Sierra::DB::Query.write_results(outfile, + results: results, format: format, + include_headers: include_headers) + end + + # Mail an existing file as an attachment. + # + # @param [String] outfile path/name of the existing file + # @param [Hash] mail_details email properties + # - required: :to, :from + # - optional: :cc, :subject, :body + # @param [Boolean] remove_file (default: false) delete file after mailing? + def mail_results(outfile, mail_details, remove_file: false) + Sierra::DB::Query.send_mail(outfile, mail_details, + remove_file: remove_file) + end + + def yield_email(index = nil) + emails = Sierra::DB::Query.emails + return emails[index] if index + emails['default_email'] + end + + # Stage/execute a query. + # + # The query isn't actually executed until the resulting Dataset is + # accessed. + # + # - Sets @query to the SQL query as a string. + # - Sets @results as Sequel::Dataset returned. + # + # @param [String] query A string containing either the SQL query itself + # or a path to a file consisting of the query. For example: + # "SELECT * FROM table WHERE a = 2 and b like 'thing'" + # @return [Sequel::Dataset] query results + def self.make_query(query) + @query = File.file?(query) ? File.read(query) : query + @results = DB.db[query] + end + + def self.results + @results + end + + def self.headers + results.columns + end + + # (see #write_results) + def self.write_results(outfile, results: self.results, + include_headers: true, format: :tsv) + puts 'writing results' + headers = + if include_headers + self.headers + else + '' + end + + format = format.to_sym + case format + when :tsv + write_tsv(outfile, results, headers) + when :csv + write_csv(outfile, results, headers) + when :xlsx + raise ArgumentError('writing to xlsx requires headers') if headers == '' + write_xlsx(outfile, results, headers) + end + end + + # Writes results as tsv. + # + # Delegated to by .write_results. + def self.write_tsv(outfile, results, headers) + write_csv(outfile, results, headers, col_sep: "\t") + end + + # Writes results as csv. + # + # Delegated to by .write_results. + def self.write_csv(outfile, results, headers, col_sep: ',') + outfile = File.open(outfile, 'wb') unless outfile.respond_to?(:read) + csv = CSV.new(outfile, col_sep: col_sep) + csv << headers unless headers.empty? + results.each do |record| + csv << record.values + end + outfile.close + end + + # Writes results as xlsx. + # + # Delegated to by .write_results. + # + # Windows only. Outfile path, if relative, is relative to user's + # windows home directory + def self.write_xlsx(outfile, results, headers) + begin + require 'win32ole' + rescue LoadError + puts <<~DOC + win32ole not found. writing output to .xlsx disabled. win32ole is + probably not available on linux/mac but should be part of the + standardlibrary on Windows installs of Ruby + DOC + raise + end + excel = WIN32OLE.new('Excel.Application') + excel.visible = true + workbook = excel.Workbooks.Add() + worksheet = workbook.Worksheets(1) + # find end column letter + end_col = ('A'..'ZZ').to_a[(headers.length - 1)] + # write headers + worksheet.Range("A1:#{end_col}1").value = headers.map(&:to_s) + # write data + i = 1 + results.each do |result| + i += 1 + worksheet.Range("A#{i}:#{end_col}#{i}").value = result.values + end + # save and close excel + outfilepath = outfile.gsub(/\//, '\\\\') + puts outfilepath + File.delete(outfilepath) if File.exist?(outfilepath) + workbook.saveas(outfilepath) + excel.quit + end + + # Returns cached email "address book" or reads it from 'email.secret' + # yaml file. + def self.emails + @emails ||= + begin + YAML.load_file('email.secret') + rescue Errno::ENOENT + YAML.load_file(File.join(base_dir, '/email.secret')) + end + end + + def self.emails=(hsh) + @emails = hsh + end + + # Returns cached smtp connection details or reads them from 'smtp.secret' + # yaml file. + # + # @return [Hash] smtp server connection details (address, port) + def self.smtp + @smtp ||= + begin + YAML.load_file('smtp.secret') + rescue Errno::ENOENT + YAML.load_file(File.join(base_dir, '/smtp.secret')) + end + end + + def self.smtp=(hsh) + @smtp = hsh + end + + # (see #mail_results) + def self.send_mail(outfile, mail_details, remove_file: false) + Mail.defaults do + delivery_method :smtp, address: smtp['address'], port: smtp['port'] + end + Mail.deliver do + from mail_details[:from] + to mail_details[:to] + subject mail_details[:subject] + body mail_details[:body] + + add_file outfile if outfile + end + File.delete(outfile) if remove_file + end + end + end +end diff --git a/lib/sierra_postgres_utilities/derivative_bib.rb b/lib/sierra_postgres_utilities/derivative_bib.rb new file mode 100644 index 0000000..9b811aa --- /dev/null +++ b/lib/sierra_postgres_utilities/derivative_bib.rb @@ -0,0 +1,140 @@ +module Sierra + # Derive alternate system bib record from Sierra bib + # + # For example, take a Sierra bib and attached records and derive + # Google Books MARC/marcxml. + # Or combine with an Internet Archive record to derive MARC/marcxml + # conforming to HathiTrust ingest specs. + # + # Generally this gets subclassed to provide alternate system-specific + # transformations and checks. + # + # The major processes this class allows for are: + # - Modify/combine Sierra MARC into alternate marc + # - Allow for MARC quality-checks + # - Write the alternate marc to xml + class DerivativeBib + attr_reader :warnings + + def initialize(bib_rec) + @sierra = bib_rec + end + + def bnum + @sierra.bnum + end + + # Original Sierra record's MARC + # + # @param [Boolean] quick (default: false) When false, uses bib's leader, + # controlfield, and varfield associations to create the marc. When true, + # uses prepared statements to retrieve the leader/etc. This is quicker + # retrieval but does not cache the retrieved fields, and so may + # be slower overall if you later need to access the varfields, etc. + # @return [MARC::Record] Sierra record's original MARC + def smarc(quick: false) + return @sierra.quick_marc if quick + + @sierra.marc + end + + # Retrieves cached alternate/transformed MARC, or derives alternate MARC + # from Sierra MARC and caches it. + # + # @return [MARC::Record] transformed Sierra MARC + def altmarc + @altmarc ||= get_alt_marc + end + + # Transforms Sierra's marc for a bib. + # + # This transformation is used for submitting MARC to HathiTrust for bibs + # scanned and uploaded to Internet Archive. Subclasses for other processes + # overwrite this method with their own custom transformations. + # + # @return [MARC::Record] transformed Sierra MARC + def get_alt_marc + # copy Sierra MARC + altmarc = MARC::Record.new_from_marc(smarc.to_marc) + + # delete things + altmarc.fields.delete_if { |f| f.tag =~ /001|003|9../ } + + # add things + altmarc.append( + MARC::ControlField.new('001', bnum.chop) # chop trailing 'a' + ) + altmarc.append(MARC::ControlField.new('003', 'NcU')) + # look for oclcnum in sierra marc; not altmarc where we may have + # just deleted the 001 + if smarc.m035_lacks_oclcnum? + altmarc.append(MARC::DataField.new('035', ' ', ' ', + ['a', "(OCoLC)#{smarc.oclcnum}"])) + end + altmarc.append(my955) if my955 + altmarc.sort + end + + # This is to be defined in subclasses (when a 955 is supposed to carry + # item/InternetArchive/whatever details). + def my955; end + + def warn(message) + @warnings << message + puts "#{bnum}\t#{message}\n" + end + + # Calls .acceptable_marc? and returns whether checks succeeded or failed. + # + # @return [Boolean] whether MARC checks passed or not + def acceptable_marc? + self.class.acceptable_marc?(altmarc) + end + + # Stub to be overwritten by subclass to perform any necessary marc + # (and/or record) checks + # + # example check: + # if @smarc.no_leader? + # warn('This bib record has no Leader. A Leader field is required. ' \ + # 'Report to cataloging staff to add Leader to record.') + # end + # + # @return [Boolean] whether MARC checks passed or not + def self.acceptable_marc?(_) + true + end + + # Conditionally writes altmarc as xml to file. + # + # @param [#read] outfile open io object for marcxml output + # @param [Boolean] strict (default: true) When true, xml is only + # written if #acceptable_marc? is true for the record. when false, the + # acceptable_marc? check is skipped and xml is written regardless. + # @param [Boolean] strip_datafields (default: true) whether datafields (not + # controlfields) should have leading/trailing whitespace stripped + # @return void + def write_xml(outfile:, strict: true, strip_datafields: true) + return if strict && !acceptable_marc? + + ofile = + if outfile.respond_to?(:write) + outfile + else + File.open(outfile, 'w') + end + + ofile.write(xml(strip_datafields: strip_datafields)) + end + + # Uses Marc::Record extension .xml_string to transform altmarc into an xml + # string. + # + # @param [Boolean] strip_datafields (default: true) whether datafields (not + # controlfields) should have leading/trailing whitespace stripped + # @return [String] altmarc as xml string + def xml(strip_datafields: true) + altmarc.xml_string(strip_datafields: strip_datafields) + end + end +end diff --git a/lib/sierra_postgres_utilities/derivative_record.rb b/lib/sierra_postgres_utilities/derivative_record.rb deleted file mode 100644 index 173b79e..0000000 --- a/lib/sierra_postgres_utilities/derivative_record.rb +++ /dev/null @@ -1,167 +0,0 @@ -# Derive alternate system bib record from Sierra bib -# -# For example, take a Sierra bib and attached records and derive -# Google Books MARC/marcxml. -# Or combine with an Internet Archive record to derive MARC/marcxml -# conforming to HathiTrust ingest specs. -# -# Generally this gets subclassed to provide alternate system-specific -# transformations and checks. -# -# The major processes this class does are: -# Modify/combine Sierra MARC into alternate marc -# Allow for MARC quality-checks -# Write the alternate marc to xml -class DerivativeRecord - attr_reader :bnum, :warnings, :sierra, :smarc - - def initialize(sierra_bib) - @warnings = [] - @sierra = sierra_bib - @bnum = @sierra.bnum - if @sierra.record_id.nil? - warn('No record was found in Sierra for this bnum') - return - elsif @sierra.deleted? - warn('Sierra bib for this bnum was deleted') - return - end - @smarc = @sierra.marc - end - - def altmarc - @altmarc ||= get_alt_marc - end - - # default marc transformation before export - # suitable for hathi and google marcxml - # otherwise, have a subclass overwrite it - - def get_alt_marc - - # copy Sierra MARC - altmarc = MARC::Record.new_from_marc(@smarc.to_marc) - - # delete things - altmarc.fields.delete_if { |f| f.tag =~ /001|003|9../ } - - # add things - altmarc.append( - MARC::ControlField.new('001', @bnum.chop) # chop trailing 'a' - ) - altmarc.append(MARC::ControlField.new('003', 'NcU')) - # look for oclcnum in sierra marc; not altmarc where we may have e.g. - # just deleted the 001 - if @smarc.m035_lacks_oclcnum? - altmarc.append(MARC::DataField.new('035', ' ', ' ', - ['a', "(OCoLC)#{@smarc.oclcnum}"])) - end - altmarc.append(my955) if my955 - altmarc.sort - end - - # This is to be defined in subclasses (when a 955 is supposed to carry - # item/InternetArchive/whatever details) - def my955 - end - - def warn(message) - @warnings << message - # if given garbage bnum, we want that to display in error - # log rather than nothing - bnum = @bnum || @sierra.given_bnum - puts "#{bnum}\t#{message}\n" - end - - # stub to be overwritten by subclass - # perform any necessary marc (or record) checks - # - # example check: - # if @smarc.no_leader? - # warn('This bib record has no Leader. A Leader field is required. Report to cataloging staff to add Leader to record.') - # end - - def check_marc; end - - # Manually writes xml, with "sensible" whitespacing. - # whitespace in text nodes retained - # linebreaks added to make human readable - # I believe options for in-built readers we tried were - # either/or in those areas. - # - # Writes the MARC faithfully, except: - # datafields (not controlfields) are stripped of leading/trailing whitespace - # drops any 002/004/009 fields - # drops any datafields containing no subfields - # xml escapes reserved characters - # - # outfile: open outfile for marcxml - # strict: - # true: perform any tests in check_marc and abort writing - # unless tests pass - # false: skip check_marc; never abort - def manual_write_xml(outfile:, strict: true, strip_datafields: true) - if strict - check_marc - return unless @warnings.empty? - end - - ofile = - if outfile.respond_to?(:write) - outfile - else - File.open(outfile, 'w') - end - - # strict is false here; we don't need to check_marc again since we - # had the opportunity above. - ofile.write(xml(strict: false, strip_datafields: strip_datafields)) - end - - def xml(strict: true, strip_datafields: true) - if strict - check_marc - return unless @warnings.empty? - end - - xml = '' - marc = altmarc.to_a - xml << "\n" - xml << " #{altmarc.leader}\n" if altmarc.leader - marc.each do |f| - if f.tag =~ /^00/ - # only process /^00[135678]/ as control fields - next if f.tag =~ /[249]$/ # drop these fields entirely - - data = escape_xml_reserved(f.value) - xml << " #{data}\n" - else - # Don't write datafields where no subfield exists. - # Note: This is not skipping fields with >= a single empty subfield - # e.g. not skipping "=856 42|u" - # This is skipping fields with no subfield - # e.g. skipping "=856 42|" and "=856 42" - next if f.subfields.empty? - - xml << " \n" - f.subfields.each do |sf| - data = escape_xml_reserved(sf.value) - data.strip! if strip_datafields - xml << " #{data}\n" - end - xml << " \n" - end - end - xml << "\n" - end - - def escape_xml_reserved(data) - return data unless data =~ /[<>&"']/ - data. - gsub('&', '&'). - gsub('<', '<'). - gsub('>', '>'). - gsub('"', '"'). - gsub("'", ''') - end -end diff --git a/lib/sierra_postgres_utilities/helpers.rb b/lib/sierra_postgres_utilities/helpers.rb deleted file mode 100644 index 7aa5ae6..0000000 --- a/lib/sierra_postgres_utilities/helpers.rb +++ /dev/null @@ -1,2 +0,0 @@ -require_relative 'helpers/hathitrust.rb' -require_relative 'helpers/varfields.rb' diff --git a/lib/sierra_postgres_utilities/helpers/hathitrust.rb b/lib/sierra_postgres_utilities/helpers/hathitrust.rb deleted file mode 100644 index 013bcef..0000000 --- a/lib/sierra_postgres_utilities/helpers/hathitrust.rb +++ /dev/null @@ -1,174 +0,0 @@ -require 'net/http' - -module SierraPostgresUtilities - module Helpers - module HathiTrust - class APIQuery - attr_accessor :response - - def initialize(oclcnums: nil, isbns: nil, level: :full) - @response = nil - return unless oclcnums || isbns - @oclcnums = oclcnums&.compact - @isbns = isbns&.compact - return if @oclcnums.empty? && isbns.empty? - @url = api_query_url(oclcnums: @oclcnums, isbns: @isbns, level: level) - @response = APIResponse.new(make_query(@url)) - end - - def marc #fulltext - @response&.fulltext_records&.map { |r| r.marc } - end - - def urls #fulltext - @response&.fulltext_records&.map { |r| r.url } - end - - def make_query(url) - return unless url - JSON.parse(Net::HTTP.get(URI.parse(URI.encode(url)))) - end - - def api_query_url(oclcnums: nil, isbns: nil, level: :brief) - base = "https://catalog.hathitrust.org/api/volumes/#{level}/json/" - terms = [] - oclcnums&.each do |num| - terms << "oclc:#{num}" - end - isbns&.each do |num| - terms << "isbn:#{num}" - end - base + terms.join("|") - end - - end - - class APIResponse - attr_accessor :json - - # The endpoint we use to search oclc/isbn identifiers - # e.g. https://catalog.hathitrust.org/api/volumes/full/json/oclc:85182211|isbn:1234567890 - # returns a response where response[identifier] == "Results hash" - # Like: - # { "oclc:85182211" => {records: [...], items: [...]}, - # "isbn:1234567890" => ... } - - # Some other endpoints return these "results hashes" directly. - # e.g. https://catalog.hathitrust.org/api/volumes/full/oclc/124815.json - # Results hash looks like: - # { - # records: [ - # {recnum: rec_details} - # ], - # items: [ - # {0: item_details}, - # {1: item_details}, - # ] - # } - # - # The records array may often contain only one record, but that is - # presumably not always the case. Items are linked to records via - # item_details[:fromRecord] - - # APIResponse is meant to be able to parse out the records and their - # attached items from either response type. - - def initialize(json) - @json = json - end - - def records - @records ||= get_records - end - - def get_records - records = {} - if @json["records"] - json = {key: @json} - else - json = @json - end - json.each do |k,v| - recs = v["records"]&.map { |r| [r[0], HathiBib.new(r[1])] }.to_h - items(v).each do |item| - next unless item.fulltext? - recs[item.recnum].items << item - end - records.merge!(recs) - end - records - end - - def fulltext_records - records.reject { |k,v| v.items.empty? }.values - end - - def items(identifier_json) - identifier_json["items"]&.map { |r| HathiItem.new(r) } - end - end - - # A Hathi "Record"/bib record - class HathiBib - attr_accessor :fulltext_items, :json - def initialize(json) - @json = json - @fulltext_items = [] - end - - def marc - io = StringIO.new(@json["marc-xml"]) - rec = MARC::XMLReader.new(io).to_a.first - add_856s(rec, @fulltext_items) - end - - def add_856s(rec, items) - return rec if items.empty? - if items.length > 1 - rec.append(rec_856) - else - rec.append(items.first.item_856) - end - rec - end - - def url - return nil if @fulltext_items.empty? - if items.length > 1 - @json["recordURL"] - else - @fulltext_items.first.json["itemURL"] - end - end - - def rec_856 - MARC::DataField.new('856', '4', '0', ['u', @json["recordURL"]], - ['y', 'Full text available via HathiTrust'], - ['3', 'Multiple volumes']) - end - end - - # A Hathi item record - class HathiItem - attr_accessor :json - - def initialize(json) - @json = json - end - - def recnum - @json["fromRecord"] - end - - def fulltext? - @json["usRightsString"] == "Full view" - end - - def item_856 - MARC::DataField.new('856', '4', '0', ['u', @json["itemURL"]], - ['y', 'Full text available via HathiTrust']) - end - end - end - end -end diff --git a/lib/sierra_postgres_utilities/helpers/varfields.rb b/lib/sierra_postgres_utilities/helpers/varfields.rb deleted file mode 100644 index 7ce7424..0000000 --- a/lib/sierra_postgres_utilities/helpers/varfields.rb +++ /dev/null @@ -1,100 +0,0 @@ -module SierraPostgresUtilities - module Helpers - module Varfields - - # returns an array - # where each element of the array - # is a hash of one row of the query results, with an added extracted content - # field - # - # 'tags' contains marc fields and associated subfields to be retrieved - # it can be a string of a single tag (e.g '130' or '210abnp') - # or an array of tags (e.g. ['130', '210abnp']) - # if no subfields are listed, all subfields are retrieved - # tag should consist of three characters, so '020' and never '20' - def get_varfields(tags) - tags = [tags] unless tags.is_a?(Array) - makedict = {} - tags.each do |entry| - m = entry.match(/^(?[0-9]{3})(?.*)$/) - marc_tag = m['tag'] - subfields = m['subfields'] unless m['subfields'].empty? - if makedict.include?(marc_tag) - makedict[marc_tag] << subfields - else - makedict[marc_tag] = [subfields] - end - end - tags = makedict - tag_phrase = tags.map { |x| "'#{x[0].to_s}'" }.join(', ') - query = <<-SQL - select * from sierra_view.varfield v - where v.record_id = #{record_id} - and v.marc_tag in (#{tag_phrase}) - order by marc_tag, occ_num - SQL - SierraDB.make_query(query) - return nil if SierraDB.results.entries.empty? - varfields = SierraDB.results.entries - varfields.each do |varfield| - varfield['extracted_content'] = [] - subfields = tags[varfield['marc_tag']] - subfields.each do |subfield| - varfield['extracted_content'] << extract_subfields( - varfield['field_content'], subfield, trim_punct: true - ) - end - end - varfields - end - - def add_explicit_sf_a(field_content) - unless field_content.chr == '|' - field_content = field_content.clone.insert(0, '|a') - end - field_content - end - - def subfield_from_field_content(field_content, subfield_tag, implicit_sfa: true) - # returns first of a given subfield from varfield field_content string - field_content = add_explicit_sf_a(field_content) if implicit_sfa - subfields = field_content.split('|') - sf_hash = {} - subfields.each do |sf| - sf_hash[sf[0]] = sf[1..-1] unless sf_hash.include?(sf[0]) - end - sf_hash[subfield_tag] - end - - def extract_subfields(field_content, desired_subfields, trim_punct: false, - remove_sf6880: true, implicit_sfa: true) - field_content = add_explicit_sf_a(field_content) if implicit_sfa - field = field_content.dup - desired_subfields ||= '' - desired_subfields = desired_subfields.join if desired_subfields.is_a?(Array) - # Remove any content preceding a pipe/subfield-delimiter - field.gsub!(/^[^|]*/, '') - field.gsub!(/\|6880[^|]*/, '') if remove_sf6880 - field.gsub!(/\|[^#{desired_subfields}][^|]*/, '') unless desired_subfields.empty? - extraction = field.gsub(/\|./, ' ').lstrip - extraction.sub!(/[.,;: \/]*$/, '') if trim_punct - extraction - end - - # field_content: "|aIDEBK|beng|erda|cIDEBK|dCOO" - # returns: - # [["a", "IDEBK"], ["b", "eng"], ["e", "rda"], ["c", "IDEBK"], ["d", "COO"]] - def subfield_arry(field_content, implicit_sfa: true) - field_content = add_explicit_sf_a(field_content) if implicit_sfa - arry = field_content.split('|') - - # delete anything prior to the first subfield delimiter (which often - # but not always means deleting an empty string), then delete - # any/other empty strings - arry.shift - arry.delete(''.freeze) - arry.map { |x| [x[0], x[1..-1]] } - end - end - end -end diff --git a/lib/sierra_postgres_utilities/hold.rb b/lib/sierra_postgres_utilities/hold.rb deleted file mode 100644 index 29c984d..0000000 --- a/lib/sierra_postgres_utilities/hold.rb +++ /dev/null @@ -1,49 +0,0 @@ -# coding: utf-8 -class SierraHold - attr_accessor :id - - include SierraPostgresUtilities::Views::Hold - - def initialize(id) - @id = id - end - - def self.status_desc(status_code) - case status_code - when '0' - 'On hold.' - when 'i', 'b', 'j' - 'Ready for pickup.' - when 't' - 'In transit to pickup.' - end - end - - def status_desc - self.class.status_desc(hold.status) - end - - def placed_date - @placed_date ||= hold.placed_gmt - end - - def object # the item/bib/volume on hold - @object ||= SierraRecord.from_id(hold.record_id) - end - - # 'item' or 'bib'; volume holds return nil - def type - object&.sql_name - end - - def patron - @patron ||= SierraPatron.from_id(hold.patron_record_id) - end - - # Returns each hold in SierraDB as a SierraHold - def self.each - SierraDB.send(:"hold"). - map{ |r| SierraHold.new(r.id)}. - each - end -end diff --git a/lib/sierra_postgres_utilities/logging.rb b/lib/sierra_postgres_utilities/logging.rb new file mode 100644 index 0000000..6ce6af9 --- /dev/null +++ b/lib/sierra_postgres_utilities/logging.rb @@ -0,0 +1,58 @@ +require 'logger' + +module Sierra + # Logging utilities. + module Logging + def logger + Logging.logger + end + + def log_to(log) + Logging.logger = + if log.respond_to?(:warn) + log + else + Logging.make_log(log) + end + end + + # Turns on (or toggles) logging of SQL queries. + # + # @param [Boolean] log_sql whether to log all SQL queries + # @return void + def log_sql(log_sql = true) + if log_sql + Sierra::DB.db.loggers << Sierra::Logging.logger + else + Sierra::DB.db.loggers.delete(Sierra::Logging.logger) + end + end + + # Retrieves cached logger or creates a new one. + def self.logger + @logger ||= make_log + end + + # Sets log to already-instantiated log. + def self.logger=(log) + @logger = log + end + + # Instantiates a log. + # + # @param [String, #write] file write log to this file / STDOUT / IO object + # @return [Logger] + def self.make_log(file = STDOUT) + log = Logger.new(file) + log.level = Logger::INFO + log.datetime_format = '%Y-%m-%d %H:%M:%S%z' + log.progname = 'sierra-postgres-utilities' + log.formatter = proc do |severity, datetime, progname, msg| + "#{datetime}: #{severity} [#{progname}] #{msg}\n" + end + log + end + end + + extend Logging +end diff --git a/lib/sierra_postgres_utilities/record.rb b/lib/sierra_postgres_utilities/record.rb new file mode 100644 index 0000000..000017d --- /dev/null +++ b/lib/sierra_postgres_utilities/record.rb @@ -0,0 +1,73 @@ +module Sierra + module Record + # Error returned when trying to retrieve a record that does not exist amd + # has never existed. + class InvalidRecord < StandardError + end + + module Factory + # Retrieve a record by rnum or id. + # + # @param [String] rnum (e.g. "b7120490a", "i1096023a") + # - The leading rec_type letter must be present. + # - The check digit must be removed. + # - The trailing "a" is technically optional. + # @param [String, Bignum] id optionally fetch by Sierra record id + # (e.g. 420908165017) + # @raise [Sierra::Record::InvalidRecord] if there is no matching record. + def get(rnum = nil, id: nil) + Factory.get(rnum: rnum, id: id) + end + + # Standardizes rnum + # + # Strips leading/trailing whitespace; ensures trailing "a". + # + # @param [String] rnum + # @return [String] standardized form of rnum (e.g. "b7120490a") + def self.standardize_rnum(rnum) + rnum = rnum.dup + rnum.strip! + unless rnum =~ /^[abciop][0-9]+a?$/ + raise InvalidRecord, "There is no record matching rnum: #{rnum}" + end + rnum << 'a' unless rnum.end_with?('a') + rnum + end + + # (see Sierra::Record.get) + def self.get(rnum:, id:) + if rnum + rnum = standardize_rnum(rnum) + rec = + case rnum.chr + when 'b' + Sierra::Data::Bib.by_record_num(record_num: rnum[1..-2]) + when 'i' + Sierra::Data::Item.by_record_num(record_num: rnum[1..-2]) + when 'c' + Sierra::Data::Holdings.by_record_num(record_num: rnum[1..-2]) + when 'o' + Sierra::Data::Order.by_record_num(record_num: rnum[1..-2]) + when 'a' + Sierra::Data::Authority.by_record_num(record_num: rnum[1..-2]) + when 'p' + Sierra::Data::Patron.by_record_num(record_num: rnum[1..-2]) + end + (rec || + # If rec is nil at this point, we still need to make sure + # it's not a deleted record or a record type not included + # in the case statement. + Sierra::Data::Metadata.first(record_type_code: rnum.chr, + record_num: rnum[1..-2])&.record || + (raise InvalidRecord, "There is no record matching rnum: #{rnum}")) + elsif id + Sierra::Data::Metadata.by_id(id: id)&.record || + (raise InvalidRecord, "There is no record matching id: #{id}") + end + end + end + + extend Factory + end +end diff --git a/lib/sierra_postgres_utilities/records.rb b/lib/sierra_postgres_utilities/records.rb deleted file mode 100644 index 95ebe6e..0000000 --- a/lib/sierra_postgres_utilities/records.rb +++ /dev/null @@ -1,7 +0,0 @@ -require_relative 'records/record' -require_relative 'records/authority' -require_relative 'records/bib' -require_relative 'records/item' -require_relative 'records/holdings' -require_relative 'records/order' -require_relative 'records/patron' diff --git a/lib/sierra_postgres_utilities/records/authority.rb b/lib/sierra_postgres_utilities/records/authority.rb deleted file mode 100644 index a98ca1e..0000000 --- a/lib/sierra_postgres_utilities/records/authority.rb +++ /dev/null @@ -1,21 +0,0 @@ -# coding: utf-8 -require_relative 'record' - -class SierraAuthority < SierraRecord - attr_reader :anum, :given_anum, :suppressed - - include SierraPostgresUtilities::Views::Authority - - @rtype = 'a' - @sql_name = 'authority' - - def initialize(rnum) - super(rnum: rnum, rtype: rtype) - @anum = @rnum - end - - def suppressed? - authority_record.is_suppressed - end - -end diff --git a/lib/sierra_postgres_utilities/records/bib.rb b/lib/sierra_postgres_utilities/records/bib.rb deleted file mode 100644 index 42772bd..0000000 --- a/lib/sierra_postgres_utilities/records/bib.rb +++ /dev/null @@ -1,364 +0,0 @@ -# coding: utf-8 - -require_relative 'record' -require 'marc' -require_relative '../../../ext/marc/record' -require_relative '../../../ext/marc/datafield' - -class SierraBib < SierraRecord - attr_reader :bnum, :given_bnum, :multiple_LDRs_flag - attr_accessor :stub, :items, :marc - - include SierraPostgresUtilities::Views::Bib - - @rtype = 'b' - @sql_name = 'bib' - -# Must be given a bnum that does not include an actual check digit. -# Good: 'b1094852a', 'b1094852' -# Bad: 'b10948521' -# If all goes well, creates a SierraBib object like so: -# "420907889860", ... } -# @given_bnum="b1094852a", -# @bnum="b1094852a" -# > - def initialize(rnum) - super(rnum: rnum, rtype: rtype) - @given_bnum = @given_rnum - @bnum = @rnum - end - - # @bnum = b1094852a - # bnum_trunc = b1094852 - def bnum_trunc - rnum_trunc - end - - # @bnum = b1094852a - # bnum_with_check = b10948521 - def bnum_with_check - rnum_with_check - end - - # not the same value as iii's is_suppressed SQL field which - # does not consider 'c' a suppression bcode3 - def suppressed? - %w[d n c].include?(bib_record[:bcode3]) - end - - # Returns cate_date as Time object - def cat_date - bib_record[:cataloging_date_gmt] - end - - # deprecated - def get_marc_varfields - marc_varfields - end - - # Returns all of a record's control fields - def control_fields - @control_fields ||= compile_control_fields - end - - # Returns all MARC control fields as array of sql result hashes - # Gathers 006/007/008 stored in control_field and any 00X in varfield) - # Fields in control_field are given proper marc_tag and field_content entries - # e.g. - # [{:id=>"63824254", ... :marc_tag=>"001", ... :field_content=>"830511"}, - # {:id=>"63824283", ... :marc_tag=>"003", ... :field_content=>"OCoLC"}, - # {:id=>"90120881", ... :control_num=>"8", :p00=>"7", ...:marc_tag=>"008", - # :field_content=>"740314c19719999oncqr4p s f0 a0eng d"}] - def compile_control_fields - return unless record_id - control = marc_varfields. - select { |tag, _| tag =~ /^00/ }. - values.flatten - control_field.each do |cf| - control_num = cf.control_num - next unless control_num.between?(6, 8) - marc_tag = "00#{control_num}" - cf = cf.to_h - # specs contain stripping logic - value = cf.values[4..43].map(&:to_s).join - value = value[0..17] if control_num == 6 - value.rstrip! if control_num == 7 - cf[:marc_tag] = marc_tag - cf[:field_content] = value - control << cf - end - control - end - - # returns leader as string - # nil when no leader field - # No bibs had >1 leader in oct 2018. We make an assumption it's not - # possible. - def ldr - @ldr ||= ldr_data_to_string(leader_field) - end - - - def rec_type #LDR/06 - leader_field.record_type_code - end - - def blvl #LDR/07 - leader_field.bib_level_code - end - - def ctrl_type #LDR/08 - leader_field.control_type_code - end - - def ldr_data_to_string(myldr) - return unless myldr.any? - - # harcoded values are default/fake values - # ldr building logic from: - # https://github.com/trln/extract_marcxml_for_argot_unc/blob/master/marc_for_argot.pl - @ldr = [ - '00000'.freeze, # rec_length - myldr.record_status_code, - myldr.record_type_code, - myldr.bib_level_code, - myldr.control_type_code, - myldr.char_encoding_scheme_code, - '2'.freeze, # indicator count - '2'.freeze, # subf_ct - myldr.base_address.to_s.rjust(5, '0'), - myldr.encoding_level_code, - myldr.descriptive_cat_form_code, - myldr.multipart_level_code, - '4500'.freeze #ldr_end - ].join - end - - def bcode1_blvl - # this usually, but not always, is the same as LDR/07(set as @blvl) - # and in cases where they do not agree, it has seemed that - # MAYBE bcode1 is more accurate and iii failed to update the LDR/07 - bib_record[:bcode1] - end - - def mat_type - bib_record[:bcode2] - end - - # Returns array of strings - # excludes "multi" as a location - # e.g. ["dd", "tr"] - def bib_locs - bib_record_location.map { |r| r.location_code }. - reject { |x| x == 'multi' } - end - - # returns iii-determines best title - # by descending preference: - # The first t-tagged MARC 245 field; 245$abghnp - # The first t-tagged non-MARC field. - # The first t-tagged MARC field other than 245 - # ( any subfields indexed for the t index display.) - # - # e.g. "Something else : a novel" - def best_title - bib_record_property.best_title - end - - # returns iii-determines best author - # by descending preference: - # The first a-tagged MARC 1XX field. - # The first b-tagged MARC 7XX field. - # The first a-tagged MARC 7XX field. - # The first a-tagged non-MARC field. - # - # e.g. "Fassnidge, Virginia." - def best_author - bib_record_property.best_author - end - - # Returns cleaned value of first 260/264 field - # e.g. "London : Constable, 1981." - def imprint - field_content = (marc_varfields['260'].to_a + marc_varfields['264'].to_a). - sort_by { |f| f[:occ_num] }. - first[:field_content] - extract_subfields(field_content, nil) - end - - def mrk - marc.to_mrk - end - - def marc - @marc ||= get_marc - end - - def get_marc - m = MARC::Record.new - m.leader = ldr if ldr - - - # add control fields stored in control_field or varfield - control_fields.each do |cf| - m << MARC::ControlField.new(cf[:marc_tag], cf[:field_content]) - end - - # add datafields stored in varfield - datafields = - marc_varfields. - reject { |tag, _| tag =~ /^00/ }. - values. - flatten - datafields.each do |vf| - m << MARC::DataField.new(vf[:marc_tag], vf[:marc_ind1], vf[:marc_ind2], - *subfield_arry(vf[:field_content].strip)) - end - m - end - - # deprecated - # returns [008/35-37, full language name] - # if invalid language code, returns [008/35-37, nil] - def lang008 - @marc.language_from_008 - end - - def oclcnum - # This method allows us to get oclcnum without doing - # any kind of explicit find_oclcnum first - # We could also set the oclcnum manually and have that - # given value returned - @oclcnum ||= marc.oclcnum - end - - def stub - return @stub if @stub - @stub = MARC::Record.new - @stub << MARC::DataField.new('907', ' ', ' ', ['a', ".#{@bnum}"]) - load_note = 'Batch load history: 999 Something records loaded 20180000, xxx.' - @stub << MARC::DataField.new('944', ' ', ' ', ['a', load_note]) - @stub - end - - # Sets and returns array of records as Sierra[Type] objects. - # empty array when none exist - - def items - @items ||= get_attached(:item, :bib_record_item_record_link) - end - - def orders - @orders ||= get_attached(:order, :bib_record_order_record_link) - end - - def holdings - @holdings ||= get_attached(:holding, :bib_record_holding_record_link) - end - - - def proper_506s(strict: true, yield_errors: false) - return unless collections.first.unl? || collections.first.sersol? - if collections.map(&:m506).map(&:to_s).uniq.count > 1 - p506s = [] - collections.each do |coll| - coll506 = coll.m506(include_sf_3: true) - next if p506s.map(&to_mrk).include?(coll506.to_mrk) - p506s << coll506 - end - else - p506s = [collections.first.m506] - end - p506s&.compact!&.sort_by!(&:to_mrk) - return p506s unless strict - errors = collections.map(&:m506_error).uniq.compact - if errors.empty? - p506s - elsif yield_errors - errors - end - end - - def correct_506s? - proper_506s == marc.fields('506').sort - end - - def extra_506s(whitelisted: []) - extra = marc.fields('506').sort - proper_506s.to_a - extra - whitelisted - end - - def lacking_506s - proper_506s.to_a - marc.fields('506').sort - end - - def collections - @collections ||= get_collections - end - - def m506_fix_output - return if correct_506s? - return unless proper_506s - lack = - if lacking_506s.empty? - nil - else - lacking_506s.map(&:to_mrk).join(';;;') - end - extra = - if extra_506s.empty? - nil - else - extra_506s.map(&:to_mrk).join(';;;') - end - [@bnum, lack, extra] - end - - def m506_error_output - errors = collections.map(&:m506_error).uniq.compact - unless errors.empty? || - (errors.count == 1 && errors.first.match(/Conc users varies by title/)) - [@bnum, errors.join(';;;')] - end - end - - def get_collections - require_relative '../../../ebook_collections' - my_colls = marc.fields('773') - my_colls.reject! do |f| - f.value =~ /^OCLC WorldShare Collection Manager managed collection/ - end - my_colls.map! { |m773| CollData.colls[m773['t']] } - my_colls.delete(nil) - @collections = my_colls - end - - def argot - require_relative '../../../../TRLN-Discovery-ETL/lib/trln_discovery_etl.rb' - @argot ||= get_argot - end - - def get_argot - @trln ||= TRLNDiscoveryRecord.new(self) - @trln.argot - end - - # Returns array of HT fulltext urls found via HT API - # - # There are only multiple urls when multiple matching - # bib records exist in HT. When a HT bib record has multiple - # items, we report the bib url, not one url for each item. - # When a HT bib record has only a single fulltext item, we - # report the item's direct url. - def ht_urls - SierraPostgresUtilities::Helpers::HathiTrust::APIQuery.new( - oclcnums: [oclcnum], - isbns: [argot["isbn"]&.map { |x| x["number"] }].flatten, - level: :brief - ).urls - end -end diff --git a/lib/sierra_postgres_utilities/records/holdings.rb b/lib/sierra_postgres_utilities/records/holdings.rb deleted file mode 100644 index 2d083c9..0000000 --- a/lib/sierra_postgres_utilities/records/holdings.rb +++ /dev/null @@ -1,37 +0,0 @@ -# coding: utf-8 -require_relative 'record' - -class SierraHoldings < SierraRecord - attr_reader :cnum, :given_cnum, :suppressed - - include SierraPostgresUtilities::Views::Holdings - - @rtype = 'c' - @sql_name = 'holding' - - def initialize(rnum) - super(rnum: rnum, rtype: rtype) - @cnum = @rnum - end - - def suppressed? - holding_record[:scode2] == 'n' - end - - # set and returns array of records as Sierra[Type] objects. - # empty array when none exist - # - - def bibs - @bibs ||= get_attached(:bib, :bib_record_holding_record_link) - end - - # holdings are attached to at most one bib - def bib - bibs.first - end - - def items - @items ||= get_attached(:item, :holding_record_item_record_link) - end -end diff --git a/lib/sierra_postgres_utilities/records/item.rb b/lib/sierra_postgres_utilities/records/item.rb deleted file mode 100644 index 15af6dd..0000000 --- a/lib/sierra_postgres_utilities/records/item.rb +++ /dev/null @@ -1,185 +0,0 @@ -# coding: utf-8 -require_relative 'record' - -class SierraItem < SierraRecord - attr_reader :inum, :given_inum - - include SierraPostgresUtilities::Views::Item - - @rtype = 'i'.freeze - @sql_name = 'item'.freeze - - # These map codes to descriptions/text. They are read live from the - # postgres DB are populated when needed. - @@itype_code_map = nil - @@location_code_map = nil - @@status_code_map = nil - - def initialize(rnum) - # Must be given an inum that does not include an actual check digit. - # Good: 'i2661010a', 'i2661010' - # Bad: 'i26610103' - # If all goes well, creates a SierraItem object like so: - # "450974227090", ...}, - # @given_inum="i2661010a", - # @inum="i2661010a"> - super(rnum: rnum, rtype: rtype) - @given_inum = @given_rnum - @inum = @rnum - end - - # @inum = i1094852a - # inum_trunc = i1094852 - def inum_trunc - rnum_trunc - end - - # @inum = i1094852a - # inum_with_check = i10948521 - def inum_with_check - rnum_with_check - end - - # array of barcode fields, nil when none exist - def barcodes(value_only: true) - varfields('b'.freeze, value_only: value_only) - end - - # array of "Library" varfields, nil when none exist - def varfield_librarys(value_only: true) - varfields('f'.freeze, value_only: value_only) - end - - # array of Stats fields, nil when none exist - def stats_fields(value_only: true) - varfields('j'.freeze, value_only: value_only) - end - - #array of message fields, nil when none exist - def messages(value_only: true) - varfields('m'.freeze, value_only: value_only) - end - - # array of volume fields, nil when none exist - def volumes(value_only: true) - varfields('v'.freeze, value_only: value_only) - end - - # array of internal notes fields, nil when none exist - def internal_notes(value_only: true) - varfields('x'.freeze, value_only: value_only) - end - - # array of public_notes fields, nil when none exist - def public_notes(value_only: true) - varfields('z'.freeze, value_only: value_only) - end - - # array of call number fields, nil when none exist - # subfield delimiters are stripped unless keep_delimiters: true - # e.g. - # item.callnos => ["PR6056.A82 S6"] - # item.callnos(keep_delimiters: true) => ["|aPR6056.A82 S6"] - # item.callnos(value_only: false) => [{ - # :id=>"8978779", :record_id=>"450974227090", :varfield_type_code=>"c", - # :marc_tag=>"090", :marc_ind1=>" ", :marc_ind2=>" ", :occ_num=>"0", - # :field_content=>"|aPR6056.A82 S6" - # }] - def callnos(value_only: true, keep_delimiters: false) - data = varfields('c'.freeze, value_only: value_only) - if value_only && !keep_delimiters - data&.map { |x| x.gsub(/\|./, '').strip } - else - data - end - end - - def icode2 - item_record[:icode2] - end - - def itype_code - item_record[:itype_code_num].to_s - end - - def location_code - item_record[:location_code] - end - - def status_code - item_record[:item_status_code] - end - - def copy_num - item_record[:copy_num] - end - - def suppressed? - item_record[:is_suppressed] - end - - def checked_out? - checkout.any? - end - - def due_date - return unless checked_out? - checkout.due_gmt - end - - def itype_description - self.class.load_itype_descs unless @@itype_code_map - @@itype_code_map[itype_code] - end - - def location_description - self.class.load_location_descs unless @@location_code_map - @@location_code_map[location_code] - end - - def status_description - self.class.load_status_descs unless @@status_code_map - @@status_code_map[status_code].capitalize - end - - def self.load_itype_descs - @@itype_code_map = SierraDB.itype_property_myuser. - map { |x| [x.code.to_s, x.name] }. - sort_by { |x| x.first.to_i }. - to_h - end - - def self.load_location_descs - @@location_code_map = SierraDB.location_myuser. - map { |x| [x.code, x.name] }. - sort. - to_h - end - - def self.load_status_descs - @@status_code_map = SierraDB.item_status_property_myuser. - map { |x| [x.code, x.name] }. - sort. - to_h - end - - # set and returns array of records as Sierra[Type] objects. - # empty array when none exist - # - - def bibs - @bibs ||= get_attached(:bib, :bib_record_item_record_link) - end - - def holdings - @holdings ||= get_attached(:holding, :holding_record_item_record_link) - end - - def is_oca? - stats_fields.any? { |x| x.match(/OCA electronic (?:book|journal)/i) } - end -end diff --git a/lib/sierra_postgres_utilities/records/order.rb b/lib/sierra_postgres_utilities/records/order.rb deleted file mode 100644 index fe5c185..0000000 --- a/lib/sierra_postgres_utilities/records/order.rb +++ /dev/null @@ -1,57 +0,0 @@ -# coding: utf-8 -require_relative 'record' - -class SierraOrder < SierraRecord - attr_reader :onum, :given_onum, :suppressed - - include SierraPostgresUtilities::Views::Order - - @rtype = 'o' - @sql_name = 'order' - - def initialize(rnum) - super(rnum: rnum, rtype: rtype) - @onum = @rnum - end - - def suppressed? - ocode3 == 'n' - end - - def status_code - order_record[:order_status_code] - end - - def ocode3 - order_record[:ocode3] - end - - def received_date - order_record[:received_date_gmt] - end - - def cat_date - order_record[:catalog_date_gmt] - end - - def location - order_record_cmf.map { |e| e[:location_code] } - end - - def number_copies - order_record_cmf.map { |e| e[:copies] } - end - - # set and returns array of records as Sierra[Type] objects. - # empty array when none exist - # - - def bibs - @bibs ||= get_attached(:bib, :bib_record_order_record_link) - end - - # orders are attached to at most one bib - def bib - bibs.first - end -end diff --git a/lib/sierra_postgres_utilities/records/patron.rb b/lib/sierra_postgres_utilities/records/patron.rb deleted file mode 100644 index b9258f4..0000000 --- a/lib/sierra_postgres_utilities/records/patron.rb +++ /dev/null @@ -1,81 +0,0 @@ -# coding: utf-8 -require_relative 'record' - -class SierraPatron < SierraRecord - attr_reader :pnum, :given_pnum - - include SierraPostgresUtilities::Views::Patron - - @rtype = 'p' - @sql_name = 'patron' - - # These map codes to descriptions/text. They are read live from the - # postgres DB are populated when needed. - @@ptype_code_map = nil - - def initialize(rnum) - super(rnum: rnum, rtype: rtype) - @pnum = @rnum - end - - def ptype_code - patron_record[:ptype_code] - end - - def pcode3 - patron_record[:pcode3_code] - end - - def expired? - expiration_date <= Date.today - end - - def expiration_date - patron_record[:expiration_date_gmt] - end - - def emails(value_only: true) - varfields('z', value_only: value_only) - end - - def email - emails.first - end - - def barcodes(value_only: true) - varfields('b', value_only: value_only) - end - - def barcode - barcodes.first - end - - # Lastname Firstname Middlename Suffix - def fullname_concat_reverse - f = patron_record_fullname.first - return unless f - "#{f[:last_name]} #{f[:first_name]} #{f[:middle_name]} #{f[:suffix_name]}". - gsub(/\s+/, ' '). - strip - end - - # Firstname Middlename Lastname Suffix - def fullname_concat - f = patron_record_fullname.first - return unless f - "#{f[:first_name]} #{f[:middle_name]} #{f[:last_name]} #{f[:suffix_name]}". - gsub(/\s+/, ' '). - strip - end - - def ptype_description - self.class.load_ptype_descs unless @@ptype_code_map - @@ptype_code_map[ptype_code] - end - - def self.load_ptype_descs - @@ptype_code_map = SierraDB.ptype_property_myuser. - map { |x| [x.value, x.name] }. - to_h - end -end diff --git a/lib/sierra_postgres_utilities/records/record.rb b/lib/sierra_postgres_utilities/records/record.rb deleted file mode 100644 index 205ddce..0000000 --- a/lib/sierra_postgres_utilities/records/record.rb +++ /dev/null @@ -1,296 +0,0 @@ -# coding: utf-8 -require_relative '../sierradb' - -class SierraRecord - attr_reader :rnum, :given_rnum, :deleted, :warnings - - include SierraPostgresUtilities::Views::Record - include SierraPostgresUtilities::Helpers::Varfields - - def self.rtype - @rtype - end - - def self.sql_name - @sql_name - end - - def rtype - self.class.rtype || @rnum[0] - end - - def sql_name - self.class.sql_name - end - - def initialize(rnum: nil, rtype: nil) - # Must be given an rnum that does not include an actual check digit. - # Good: 'i2661010a', 'i26610102' - # Bad: 'i26610103' - # If all goes well, creates a SierraWhatever object like so: - # - rnum = rnum.strip - @given_rnum = rnum - @warnings = [] - if rnum =~ /^#{rtype}[0-9]+a?$/ - @rnum = rnum.dup - @rnum << 'a' unless rnum.end_with?('a') - else - @warnings << "Cannot retrieve Sierra record. Rnum must start with #{rtype}" - return - end - if record_id.nil? - @warnings << 'No record was found in Sierra for this record number' - elsif deleted? - @warnings << 'This Sierra record was deleted' - end - end - - # e.g. #" - def inspect - "#<#{self.class.name}:#{rnum}>" - end - - # @rnum = i1094852a - # rnum_trunc = i1094852 - def rnum_trunc - return nil unless @rnum - @rnum.chop - end - - # @rnum = i1094852a - # inum_with_check = i10948521 - def rnum_with_check - return nil unless @rnum - @rnum.chop + check_digit(recnum) - end - - # @rnum = i1094852a - # recnum = 1094852 - def recnum - return nil unless @rnum - @rnum[/\d+/] - end - - # Returns record id - # nil when record does not exist (e.g. given bad recnum) - def record_id - @record_id ||= record_metadata[:id] - end - - def check_digit(recnum) - digits = recnum.split('').reverse - y = 2 - sum = 0 - digits.each do |digit| - sum += digit.to_i * y - y += 1 - end - remainder = sum % 11 - if remainder == 10 - 'x' - else - remainder.to_s - end - end - - # Returns all varfields (non-marc and marc) - # { varfield_type_code: array of field_content(s),... } - # empty array when no such varfields - def varfields(type_or_tag = nil, value_only: false) - arry = - if type_or_tag - varfields_by_type[type_or_tag] || marc_varfields[type_or_tag] || [] - else - varfields_by_type.values.flatten - end - if value_only - arry&.map { |f| f[:field_content] } - else - arry - end - end - - # Sets/returns hash of varfields with varfield_type tags as keys - def varfields_by_type - @varfields_by_type ||= type_vf - end - - # Returns hash of varfields with varfield_type tags as keys - def type_vf - fields = varfield.sort_by { |field| - [field[:varfield_type_code], field[:occ_num], field[:id]] - } - vf = fields.group_by { |f| f[:varfield_type_code] } - vf.delete(nil) - vf - end - - # Sets/returns hash of marc varfields with marc_tag's as keys - def marc_varfields - @marc_varfields ||= marc_vf - end - - # Returns hash of marc varfields with marc_tag's as keys - def marc_vf - vf = varfield.group_by { |f| f.marc_tag } - vf.delete(nil) - vf - end - - # Returns hash of varfield_type_codes and their names - # e.g., for items, { 'b' => 'Barcode', ...} - def self.vf_codes(rtype: self.rtype) - return @vf_codes if @vf_codes - query = <<~SQL - select t.code, - case when n.name = '' then n.short_name else n.name end - from sierra_view.varfield_type t - inner join sierra_view.varfield_type_name n - on n.varfield_type_id = t.id - where t.record_type_code = '#{rtype}' - order by t.code - SQL - SierraDB.make_query(query) - @vf_codes = SierraDB.results.values.to_h - end - - def vf_codes - self.class.vf_codes(rtype: rtype) - end - - def deleted? - true if record_metadata['deletion_date_gmt'] - end - - def invalid? - true if record_id.nil? - end - - # Returns rec creation date - def created_date - record_metadata.creation_date_gmt - end - - # Returns rec updated date - def updated_date - record_metadata.record_last_updated_gmt - end - - # Returns array of attached Sierra[name] objects - # empty array when none exist - def get_attached(name, view) - send(view).map { |r| SierraRecord.from_id(r.send("#{name}_record_id")) } - end - - def self.from_id(id) - return nil unless id - values = SierraDB.conn.exec_prepared( - 'id_find_record_metadata', - [id] - ).first&.values - return nil unless values - metadata = - SierraPostgresUtilities::Views::Record.record_metadata_struct.new( - *values - ) - rm_factory(metadata) - end - - def self.rm_factory(record_metadata) - rtype = record_metadata[:record_type_code] - rnum = "#{rtype}#{record_metadata[:record_num]}a" - rec = factory(rnum, rtype: rtype) - rec.instance_variable_set("@read_record_metadata", record_metadata) - rec - end - - def self.factory(rnum, rtype: nil) - rtype = rnum[0] unless rtype - case rtype - when 'b' - SierraBib.new(rnum) - when 'i' - SierraItem.new(rnum) - when 'c' - SierraHoldings.new(rnum) - when 'o' - SierraOrder.new(rnum) - when 'a' - SierraAuthority.new(rnum) - when 'p' - SierraPatron.new(rnum) - end - end - - def self.from_phrase_search(index:, entry:) - regexp = "^#{index}#{entry.downcase}" - recs = SierraDB.conn.exec_prepared('search_phrase_entry', [regexp]). - column_values(0). - map { |id| SierraRecord.from_id(id)} - return recs if recs.length > 1 - return if recs.empty? - recs.first - end - - def self.from_create_list(listnum) - query = <<~SQL - select record_metadata_id - from sierra_view.bool_set - where bool_info_id = #{listnum} - SQL - recs = SierraDB.make_query(query). - column_values(0). - map! { |id| SierraRecord.from_id(id)} - return if recs.empty? - recs - end - - - # items = SierraItem.by_field(:location_code, 'aahd') - # items = SierraRecord.by_field(:itype_code_num, '87', sqlname: 'item') - # bibs = SierraBib.by_field(:bcode3, 'c') - def self.by_field(field, criteria, sqlname: nil) - sqlname = sqlname || sql_name - criteria = [criteria] unless criteria.is_a?(Array) - query = <<~SQL - select id - from sierra_view.#{sqlname}_record - where #{field} in ('#{criteria.join("', '")}') - SQL - SierraDB.make_query(query). - column_values(0). - map! { |rid| SierraRecord.from_id(rid) } - end - - # SierraBib.random.best_title - # SierraRecord.random(sqlname: 'item') - # SierraItem.random(limit: 10).map { |i| i.barcodes.first } - def self.random(limit: 1, sqlname: nil) - sqlname = sqlname || sql_name - query = <<~SQL - select id - from sierra_view.#{sqlname}_record - where random() < 0.01 - limit 10000 - SQL - SierraDB.make_query(query) - recs = SierraDB.results.column_values(0). - sort_by { |x| rand }[0..-1+limit]. - map! { |rid| SierraRecord.from_id(rid) } - return recs if recs.length > 1 - recs.first - end - - def self.each - SierraDB.send(:"#{sql_name}_record"). - each. - lazy. - map { |r| SierraRecord.from_id(r.id) } - end -end diff --git a/lib/sierra_postgres_utilities/search.rb b/lib/sierra_postgres_utilities/search.rb new file mode 100644 index 0000000..f296eb6 --- /dev/null +++ b/lib/sierra_postgres_utilities/search.rb @@ -0,0 +1,79 @@ +require_relative 'data/helpers/phrase_normalization.rb' + +module Sierra + module Search + module PhraseSearch + NORMALIZATION = Sierra::Data::Helpers::PhraseNormalization + + # Searches phrase_entry and returns matching records. + # + # Note: Sierra indexes an x-digit isbn as both a 10- and 13-digit isbn, + # so searching for either form of the isbn will work, even if only + # one form is recorded in the record. + # + # @param [Symbol, String] index index to search, e.g. :t, :o, :b + # @param [String] phrase search phrase + # @param [String] rec_type optionally limit results to specified rec_type, + # (e.g. 'b', 'i') + # @param [Symbol] match_type optionally specify matching strategy to use, + # :exact or :startswith + # @return [Array] array of search results as records + # (e.g. Sierra::Data::Bib) + def phrase_search(index, phrase, rec_type: nil, match_type: nil) + Sierra::Search::PhraseSearch.phrase_search( + index, phrase, rec_type: rec_type, match_type: match_type + ).lazy.map(&:record) + end + + # (see #phrase_search) + def self.phrase_search(index, phrase, + rec_type: nil, match_type: nil) + index = index&.to_sym + norm_term = NORMALIZATION.normalize(phrase, index) + return unless norm_term + + match_type ||= match_type(index) + + dataset = Sierra::Data::PhraseEntry. + where(Sequel.lit( + '(index_tag || index_entry) ' \ + "#{match_statement(match_type)}#{index}#{norm_term}'" + )) + + return dataset unless rec_type + dataset. + association_join(:record_metadata). + where(record_type_code: rec_type) + end + + # Get the default match strategy for an index. + # + # @param [Symbol] index the index + # @return [Symbol] the default match strategy + def self.match_type(index) + case index + when :n, :a, :t, :s + :startswith + else # :i, :b, :o, :c, :e, :g + :exact + end + end + private_class_method :match_type + + # @param [Symbol] match_type the match strategy to use, :exact or + # :startswith + # @return [String] SQL fragment for given match strategy + def self.match_statement(match_type) + case match_type + when :exact + " = '" + when :startswith + " ~ '^" + end + end + private_class_method :match_statement + end + + extend PhraseSearch + end +end diff --git a/lib/sierra_postgres_utilities/sierradb.rb b/lib/sierra_postgres_utilities/sierradb.rb deleted file mode 100644 index 582fd4a..0000000 --- a/lib/sierra_postgres_utilities/sierradb.rb +++ /dev/null @@ -1,264 +0,0 @@ -require 'csv' -require 'yaml' -require 'mail' -require 'pg' -# 'win32ole' conditionally loaded - - -module SierraDB - - # extended (in views.rb) by - # SierraPostgresUtilities::Views::General - # to allow almost all views to be read with SierraDB.view_name - # e.g. SierraDB.branch_myuser - - def self.initial_creds(creds = nil) - @initial_creds ||= creds - end - - def self.conn(creds: initial_creds || 'prod') - @conn ||= make_connection(creds: creds) - end - - # Re-establish a connection (with the same or alternate credentials) - def self.connect_as(creds:) - @conn.close if @conn && !@conn.finished? - @conn = make_connection(creds: creds) - end - - # Close the connection - def self.close - @conn.close - end - - # Headers for the current results - def self.headers - @results&.fields - end - - def self.results - @results - end - - def self.query - @query - end - - # query is just an SQL query as a string - # query = "SELECT * FROM table WHERE a = 2 and b like 'thing'" - # or as a file containing such a string - def self.make_query(query) - run_query(query) - end - - def self.prepare_query(name, query) - conn.prepare(name, query) - register_prepared_query(name, query) - end - - def self.register_prepared_query(name, query) - prepared_queries[name] = query - end - - def self.prepared_queries - @prepared_queries ||= {} - end - - def self.write_results(outfile, results: self.results, headers: self.headers, - include_headers: true, format: :tsv) - # needs relative path for xlsx output - puts 'writing results' - headers = '' unless include_headers - format = format.to_sym unless format.is_a?(Symbol) - if format == :tsv - write_tsv(outfile, results, headers) - elsif format == :csv - write_csv(outfile, results, headers) - elsif format == :xlsx - raise ArgumentError('writing to xlsx requires headers') if headers == '' - write_xlsx(outfile, results, headers) - end - end - - # Mail an existing file as an attachment. - # - # outfile: the existing file - # mail_details: a hash of email properties - # required - :to, :from - # optional - :cc, :subject, :body - # remove_file: delete file after sending? - def self.mail_results(outfile, mail_details, remove_file: false) - send_mail(outfile, mail_details, remove_file: remove_file) - end - - def self.emails - @emails ||= - begin - YAML.load_file('email.secret') - rescue Errno::ENOENT - YAML.load_file(File.join(base_dir, '/email.secret')) - end - end - - def self.yield_email(index = nil) - return emails[index] if index - emails['default_email'] - end - - def self.base_dir - File.dirname(File.expand_path('..', __dir__)).to_s - end - - # Connects to SierraDB using creds from specified YAML file or given hash. - # - # Possible specified credentials: - # 'prod' : uses sierra_prod.secret in pwd or base directory; creds for prod db - # 'test' : uses sierra_test.secret in pwd or base directory; creds for test db - # [filename] : reads secrets from file specified. First looks in pwd, then - # looks in base directory - # [somehash]: accepts a hash containing the credentials - def self.make_connection(creds:) - if creds == 'prod' - creds = 'sierra_prod.secret' - elsif creds == 'test' - creds = 'sierra_test.secret' - end - - if creds.is_a?(Hash) - @cred = creds - else - begin - @cred = YAML.load_file(creds) - rescue Errno::ENOENT - begin - @cred = YAML.load_file(File.join(base_dir, creds)) - rescue Errno::ENOENT - raise('Connection credentials invalid or not found.') - end - end - end - c = PG::Connection.new(@cred) - - # load any already prepared queries - prepared_queries.each do |name, statement| - c.prepare(name, statement) - end - - # set type mapping - set_type_map(c) - end - - def self.set_type_map(conn) - conn.type_map_for_results = PG::BasicTypeMapForResults.new(conn) - - # add text coders for any oids without defined cast, e.g. oid 1700 - # https://stackoverflow.com/questions/34795078/pg-gem-warning-no-type-cast-defined-for-type-numeric - text_coder = conn.type_map_for_results.coders.find { |c| c.name == 'text' } - new_coder = text_coder.dup.tap { |c| c.oid = 1700 } - conn.type_map_for_results.add_coder(new_coder) - - conn - end - - def self.write_tsv(outfile, results, headers) - write_csv(outfile, results, headers, col_sep: "\t") - end - - def self.write_csv(outfile, results, headers, col_sep: ',') - CSV.open(outfile, 'wb', col_sep: col_sep) do |csv| - csv << headers unless headers.empty? - results.each do |record| - csv << record.values - end - end - end - - def self.write_xlsx(outfile, results, headers) - begin - require 'win32ole' - rescue LoadError - puts "win32ole not found. writing output to .xlsx disabled. win32ole is - probably not available on linux/mac but should be part of the standard - library on Windows installs of Ruby" - raise - end - excel = WIN32OLE.new('Excel.Application') - excel.visible = false - workbook = excel.Workbooks.Add() - worksheet = workbook.Worksheets(1) - # find end column letter - end_col = ('A'..'ZZ').to_a[(headers.length - 1)] - # write headers - worksheet.Range("A1:#{end_col}1").value = headers - # write data - i = 1 - results.each do |result| - i += 1 - worksheet.Range("A#{i}:#{end_col}#{i}").value = result.values - end - # save and close excel - outfilepath = outfile.gsub(/\//, '\\\\') - File.delete(outfilepath) if File.exist?(outfilepath) - workbook.saveas(outfilepath) - excel.quit - end - - def self.smtp - @smtp ||= - begin - YAML.load_file('smtp.secret') - rescue Errno::ENOENT - YAML.load_file(File.join(base_dir, '/smtp.secret')) - end - end - - def self.send_mail(outfile, mail_details, remove_file: false) - smtp = SierraDB.smtp - Mail.defaults do - delivery_method :smtp, address: smtp['address'], port: smtp['port'] - end - Mail.deliver do - from mail_details[:from] - to mail_details[:to] - subject mail_details[:subject] - body mail_details[:body] - - add_file outfile if outfile - end - File.delete(outfile) if remove_file - end - - # query is just an SQL query as a string - # query = "SELECT * FROM table WHERE a = 2 and b like 'thing'" - # or as a file containing such a string - def self.run_query(query) - @query = File.file?(query) ? File.read(query) : query - @results = conn.exec(@query) - end - private_class_method :run_query - - def self.viewstruct(view_name) - query = <<~SQL - select column_name - from information_schema.columns - where table_schema = 'sierra_view' and table_name = '#{view_name}' - SQL - SierraDB.make_query(query) - fields = SierraDB.results.values.flatten.map { |s| s.to_sym } - Struct.new(*fields) - end - - def self.view_info(view_name) - query_name = 'read_view_info' - unless prepared_queries.has_key?(query_name) - statement = <<~SQL - select column_name, data_type - from information_schema.columns - where table_schema = 'sierra_view' and table_name = $1::text - SQL - self.prepare_query(query_name, statement) - end - SierraDB.conn.exec_prepared(query_name, [view_name]). - values.to_h - end -end diff --git a/lib/sierra_postgres_utilities/spec_support.rb b/lib/sierra_postgres_utilities/spec_support.rb new file mode 100644 index 0000000..3fcd250 --- /dev/null +++ b/lib/sierra_postgres_utilities/spec_support.rb @@ -0,0 +1,9 @@ +module Sierra + module SpecSupport + # Location of sierra_postgres_utilities testing factories. + # + # @example Include those factories in another app + # FactoryBot.definition_file_paths << Sierra::SpecSupport::FACTORY_PATH + FACTORY_PATH = File.expand_path('../../spec/factories', __dir__) + end +end diff --git a/lib/sierra_postgres_utilities/user.rb b/lib/sierra_postgres_utilities/user.rb deleted file mode 100644 index 9b12a96..0000000 --- a/lib/sierra_postgres_utilities/user.rb +++ /dev/null @@ -1,21 +0,0 @@ -class SierraUser - attr_accessor :login - - include SierraPostgresUtilities::Views::User - - def initialize(login) - @login = login - end - - def id - iii_user.id - end - - def permissions - iii_user_permission_myuser.map { |r| [r.permission_num, r.permission_name] } - end - - def statistic_group_code_num - iii_user.statistic_group_code_num - end -end diff --git a/lib/sierra_postgres_utilities/version.rb b/lib/sierra_postgres_utilities/version.rb index b9718b5..3fa803b 100644 --- a/lib/sierra_postgres_utilities/version.rb +++ b/lib/sierra_postgres_utilities/version.rb @@ -1,3 +1,3 @@ -module SierraPostgresUtilities - VERSION = '0.2.2'.freeze +module Sierra + VERSION = '0.3.0'.freeze end diff --git a/lib/sierra_postgres_utilities/views.rb b/lib/sierra_postgres_utilities/views.rb deleted file mode 100644 index 90f0d44..0000000 --- a/lib/sierra_postgres_utilities/views.rb +++ /dev/null @@ -1,15 +0,0 @@ -require_relative 'views/method_constructor.rb' -require_relative 'views/authority.rb' -require_relative 'views/bib.rb' -require_relative 'views/general.rb' -require_relative 'views/hold.rb' -require_relative 'views/holdings.rb' -require_relative 'views/item.rb' -require_relative 'views/order.rb' -require_relative 'views/patron.rb' -require_relative 'views/record.rb' -require_relative 'views/user.rb' - -module SierraDB - extend SierraPostgresUtilities::Views::General -end diff --git a/lib/sierra_postgres_utilities/views/authority.rb b/lib/sierra_postgres_utilities/views/authority.rb deleted file mode 100644 index d74007b..0000000 --- a/lib/sierra_postgres_utilities/views/authority.rb +++ /dev/null @@ -1,25 +0,0 @@ -module SierraPostgresUtilities - module Views - module Authority - extend Views::MethodConstructor - - views = [ - { - view: :authority_record, - view_match: :id, obj_match: :record_id, - entries: :first - }, - { - view: :authority_view, - view_match: :id, obj_match: :record_id, - entries: :first - }, - ] - - views.each do |hsh| - match_view(hsh) - access_view(hsh) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/bib.rb b/lib/sierra_postgres_utilities/views/bib.rb deleted file mode 100644 index 4d05693..0000000 --- a/lib/sierra_postgres_utilities/views/bib.rb +++ /dev/null @@ -1,73 +0,0 @@ -module SierraPostgresUtilities - module Views - module Bib - extend Views::MethodConstructor - - views = [ - { - view: :bib_record, - view_match: :id, obj_match: :record_id, - entries: :first - }, - { - view: :bib_record_call_number_prefix, - view_match: :bib_record_id, obj_match: :record_id, - entries: :first - }, - { - view: :bib_record_holding_record_link, - view_match: :bib_record_id, obj_match: :record_id, - entries: :all, sort: :holdings_display_order - }, - { - view: :bib_record_item_record_link, - view_match: :bib_record_id, obj_match: :record_id, - entries: :all, sort: :items_display_order - }, - { - view: :bib_record_location, - view_match: :bib_record_id, obj_match: :record_id, - entries: :all, sort: :display_order - }, - { - view: :bib_record_order_record_link, - view_match: :bib_record_id, obj_match: :record_id, - entries: :all, sort: :orders_display_order - }, - { - view: :bib_record_property, - view_match: :bib_record_id, obj_match: :record_id, - entries: :first - }, - { - view: :bib_record_volume_record_link, - view_match: :bib_record_id, obj_match: :record_id, - entries: :all, sort: :volumes_display_order - }, - { - view: :bib_view, - view_match: :id, obj_match: :record_id, - entries: :first - }, - { - view: :control_field, - view_match: :record_id, obj_match: :record_id, - entries: :all, - sort: %i[varfield_type_code occ_num id] - }, - { - view: :leader_field, - view_match: :record_id, obj_match: :record_id, - # No bibs had >1 leader in oct 2018. Make an assumption it's not - # possible. - entries: :first - }, - ] - - views.each do |hsh| - match_view(hsh) - access_view(hsh) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/general.rb b/lib/sierra_postgres_utilities/views/general.rb deleted file mode 100644 index 9a78d2a..0000000 --- a/lib/sierra_postgres_utilities/views/general.rb +++ /dev/null @@ -1,128 +0,0 @@ -module SierraPostgresUtilities - module Views - - - # Retrieves general views( as in the whole tables, not just entries in the - # context of a particular record) - module General - extend Views::MethodConstructor - - - # We can define views here, but the only thing it adds is sorting - # (which is of questionable value). Undefined views will still be - # made available. - DEFINED_VIEWS = [ - {view: :agency_property_myuser, sort: :display_order}, - {view: :bib_level_property_myuser, sort: :display_order}, - {view: :hold, sort: :id}, - {view: :item_status_property_myuser, sort: :display_order}, - {view: :itype_property_myuser, sort: :display_order}, - {view: :location_myuser, sort: :display_order}, - {view: :ptype_property_myuser, sort: :display_order} - ] - - # For large views we don't want to grab in their entirety, we define - # them here to get enumerators that read a chunk of the view at a time. - # Sorting matters for these views because the chunks are retrieved - # using offsets. We default sort everything by id unless otherwise - # defined, and only the subfield views lack is columns and need sorting - # specified here. - STREAMED_VIEWS = { - authority_record: nil, - authority_view: nil, - bib_record: nil, - bib_record_call_number_prefix: nil, - bib_record_holding_record_link: nil, - bib_record_item_record_link: nil, - bib_record_location: nil, - bib_record_order_record_link: nil, - bib_record_property: nil, - bib_record_volume_record_link: nil, - bib_view: nil, - bool_set: nil, - control_field: nil, - holding_record: nil, - holding_record_box: nil, - holding_record_item_record_link: nil, - holding_record_location: nil, - holding_view: nil, - invoice_record: nil, - invoice_record_line: nil, - invoice_record_vendor_summary: nil, - invoice_view: nil, - item_record: nil, - item_record_property: nil, - item_view: nil, - leader_field: nil, - order_record: nil, - order_record_cmf: nil, - order_record_paid: nil, - order_view: nil, - patron_record: nil, - patron_record_address: nil, - patron_record_fullname: nil, - patron_record_phone: nil, - patron_view: nil, - phrase_entry: nil, - reading_history: nil, - record_metadata: nil, - subfield: {sort: [:record_id, :varfield_id]}, - subfield_view: {sort: [:record_id, :varfield_id]}, - varfield: nil, - varfield_view: nil, - volume_record: nil, - volume_record_item_record_link: nil, - volume_view: nil, - } - - # Create methods for any of the explicitly DEFINED_VIEWS - # Methods for other views will be created on demand. - DEFINED_VIEWS.each do |hsh| - read_view(hsh) - access_view(hsh) - end - - # Defines methods to read/access views that don't already have methods. - def method_missing(m, *args, &block) - if STREAMED_VIEWS.has_key?(m) - hsh = {view: m} - hsh.merge!(STREAMED_VIEWS[m]) if STREAMED_VIEWS[m] - Views::General.stream_view(hsh) - return self.send(m.to_sym) - elsif views.include?(m) - hsh = {view: m} - Views::General.read_view(hsh) - Views::General.access_view(hsh) - return self.send(m.to_sym) - else - raise NoMethodError, "undefined method '#{m}'" - end - end - - def respond_to?(method_name, include_private = false) - @views.include?(method_name.to_sym) || super - end - - def views - @views ||= get_db_views - end - - # Returns array of symbolized table/view names in sierra_view - # Excludes views we've already defined or excluded - def get_db_views - query = <<~SQL - select table_name - from information_schema.views - where table_schema = 'sierra_view' - SQL - SierraDB.make_query(query) - SierraDB.results.values.flatten.map { |x| x.to_sym }.sort - end - - # refreshes a cached view that has presumably already had a method defined - def refresh_view(name) - self.instance_variable_set("@#{name}", self.send("read_#{name}")) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/hold.rb b/lib/sierra_postgres_utilities/views/hold.rb deleted file mode 100644 index 3a1f5c0..0000000 --- a/lib/sierra_postgres_utilities/views/hold.rb +++ /dev/null @@ -1,20 +0,0 @@ -module SierraPostgresUtilities - module Views - module Hold - extend Views::MethodConstructor - - views = [ - { - view: :hold, - view_match: :id, obj_match: :id, - entries: :first - } - ] - - views.each do |hsh| - match_view(hsh) - access_view(hsh) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/holdings.rb b/lib/sierra_postgres_utilities/views/holdings.rb deleted file mode 100644 index 977189f..0000000 --- a/lib/sierra_postgres_utilities/views/holdings.rb +++ /dev/null @@ -1,45 +0,0 @@ -module SierraPostgresUtilities - module Views - module Holdings - extend Views::MethodConstructor - - views = [ - { - view: :bib_record_holding_record_link, - view_match: :holding_record_id, obj_match: :record_id, - entries: :first - }, - { - view: :holding_record, - view_match: :id, obj_match: :record_id, - entries: :first - }, - { - view: :holding_record_card, - view_match: :holding_record_id, obj_match: :record_id, - entries: :all, sort: :id - }, - { - view: :holding_record_location, - view_match: :holding_record_id, obj_match: :record_id, - entries: :all, sort: :display_order - }, - { - view: :holding_record_item_record_link, - view_match: :holding_record_id, obj_match: :record_id, - entries: :all, sort: :items_display_order - }, - { - view: :holding_view, - view_match: :id, obj_match: :record_id, - entries: :first - }, - ] - - views.each do |hsh| - match_view(hsh) - access_view(hsh) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/item.rb b/lib/sierra_postgres_utilities/views/item.rb deleted file mode 100644 index 877ad9b..0000000 --- a/lib/sierra_postgres_utilities/views/item.rb +++ /dev/null @@ -1,51 +0,0 @@ - -module SierraPostgresUtilities - module Views - module Item - extend Views::MethodConstructor - - views = [ - { - view: :bib_record_item_record_link, - view_match: :item_record_id, obj_match: :record_id, - entries: :all, sort: :bibs_display_order - }, - { - view: :checkout, - view_match: :item_record_id, obj_match: :record_id, - entries: :first - }, - { - view: :hold, - view_match: :record_id, obj_match: :record_id, - entries: :all, sort: :records_display_order - }, - { - view: :holding_record_item_record_link, - view_match: :item_record_id, obj_match: :record_id, - entries: :all, sort: :holdings_display_order - }, - { - view: :item_record, - view_match: :id, obj_match: :record_id, - entries: :first - }, - { - view: :item_record_property, - view_match: :item_record_id, obj_match: :record_id, - entries: :first - }, - { - view: :item_view, - view_match: :id, obj_match: :record_id, - entries: :first - }, - ] - - views.each do |hsh| - match_view(hsh) - access_view(hsh) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/method_constructor.rb b/lib/sierra_postgres_utilities/views/method_constructor.rb deleted file mode 100644 index 40bc34f..0000000 --- a/lib/sierra_postgres_utilities/views/method_constructor.rb +++ /dev/null @@ -1,129 +0,0 @@ -module SierraPostgresUtilities - module Views - - # Creates methods to read/cache a given view either in the view's entirety - # or limited to the context of, for example, a specific record. - module MethodConstructor - - # Creates a method that retrieves an entire DB view - # - # Arguments: - # view: name of the view to be read - # (i.e. :bib_record for "sierra_view.bib_record") - # sort: field or array of fields to sort results by - def read_view(hsh) - viewstruct = SierraDB.viewstruct(hsh[:view]) - define_method("read_#{hsh[:view]}") do - query = <<~SQL - select * - from sierra_view.#{hsh[:view]} - SQL - SierraDB.make_query(query). - entries. - sort_by! { |x| x[hsh[:sort].to_s] } - SierraDB.results.values.map! { |r| viewstruct.new(*r) } - end - end - - # Create a method that returns an enumerator for larger DB views - def stream_view(hsh) - viewstruct = SierraDB.viewstruct(hsh[:view]) - sort = hsh[:sort] || [:id] - sort.flatten! - sort.compact! - statement = <<~SQL - select * from sierra_view.#{hsh[:view]} - order by #{sort.join(', ')} - limit 10000 - offset $1::int - SQL - SierraDB.prepare_query("stream_#{hsh[:view]}", statement) - - define_method("stream_#{hsh[:view]}") do - stream = Enumerator.new do |y| - n = 0 - not_done = true - while not_done - values = SierraDB.conn.exec_prepared( - "stream_#{hsh[:view]}", - [n]).values - not_done = false if values.empty? - values.each { |r| y << viewstruct.new(*r) } - n += 10000 - end - end - end - - # The enumeration isn't memoized, so we just alias, for example, - # SierraDB.stream_bib_record as SierraDB.bib_record - define_method(hsh[:view]) do - self.send("stream_#{hsh[:view]}") - end - end - - # Creates a method that retrieves matching records in a DB view. - # Generally used to scope DB results to entries that match a given - # object/record_id - # - # Returns nil unless obj_match returns truthy - # - # Arguments: - # view: name of the view to be read - # (i.e. :bib_record for "sierra_view.bib_record") - # - # view_match: field from the DB view to match on - # obj_match: object method to match on - # cast: will cast obj_match to given sql type - # (default is bigint, appropriate for record_id's) - # - # - # sort: field or array of fields to sort results by - # entries: - # when 'all': return all entries in an array - # when 'first': return first of any entries - - def match_view(hsh) - viewstruct = SierraDB.viewstruct(hsh[:view]) - cast = hsh[:cast] || :bigint - statement = <<~SQL - select * - from sierra_view.#{hsh[:view]} - where #{hsh[:view_match]} = $1::#{cast} - SQL - sort = [hsh[:sort]].flatten.compact - statement << "order by #{sort.join(', ')}" if sort.any? - name = self.name - SierraDB.prepare_query("#{name}_match_#{hsh[:view]}", statement) - - define_method("read_#{hsh[:view]}") do - obj_match = self.send(hsh[:obj_match]) - return unless obj_match - results = SierraDB.conn.exec_prepared( - "#{name}_match_#{hsh[:view]}", - [obj_match] - ).values - - case hsh[:entries] - when :all - results.map! { |r| viewstruct.new(*r) } - when :first - viewstruct.new(*results&.first) - end - end - end - - # Creates method to return cached view or retrieve view if not yet cached. - def access_view(hsh) - define_method(hsh[:view]) do - (self.instance_variable_get("@#{hsh[:view]}") || - self.instance_variable_set("@#{hsh[:view]}", self.send("read_#{hsh[:view]}"))) - end - end - - # refreshes a cached view that has presumably already had a method defined - def refresh_view(name) - self.instance_variable_set("@#{name}", self.send("read_#{name}")) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/order.rb b/lib/sierra_postgres_utilities/views/order.rb deleted file mode 100644 index a971324..0000000 --- a/lib/sierra_postgres_utilities/views/order.rb +++ /dev/null @@ -1,35 +0,0 @@ -module SierraPostgresUtilities - module Views - module Order - extend Views::MethodConstructor - - views = [ - { - view: :bib_record_order_record_link, - view_match: :order_record_id, obj_match: :record_id, - entries: :first - }, - { - view: :order_record, - view_match: :id, obj_match: :record_id, - entries: :first - }, - { - view: :order_record_cmf, - view_match: :order_record_id, obj_match: :record_id, - entries: :all, sort: :display_order - }, - { - view: :order_view, - view_match: :id, obj_match: :record_id, - entries: :first - }, - ] - - views.each do |hsh| - match_view(hsh) - access_view(hsh) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/patron.rb b/lib/sierra_postgres_utilities/views/patron.rb deleted file mode 100644 index 6a77caa..0000000 --- a/lib/sierra_postgres_utilities/views/patron.rb +++ /dev/null @@ -1,35 +0,0 @@ -module SierraPostgresUtilities - module Views - module Patron - extend Views::MethodConstructor - - views = [ - { - view: :patron_record, - view_match: :id, obj_match: :record_id, - entries: :first, - }, - { - view: :patron_record_address, - view_match: :patron_record_id, obj_match: :record_id, - entries: :all, sort: :display_order - }, - { - view: :patron_record_fullname, - view_match: :patron_record_id, obj_match: :record_id, - entries: :all, sort: :display_order - }, - { - view: :patron_record_phone, - view_match: :patron_record_id, obj_match: :record_id, - entries: :all, sort: :display_order - } - ] - - views.each do |hsh| - match_view(hsh) - access_view(hsh) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/record.rb b/lib/sierra_postgres_utilities/views/record.rb deleted file mode 100644 index 0546535..0000000 --- a/lib/sierra_postgres_utilities/views/record.rb +++ /dev/null @@ -1,73 +0,0 @@ -module SierraPostgresUtilities - module Views - module Record - extend Views::MethodConstructor - - views = [ - { - view: :phrase_entry, - view_match: :record_id, obj_match: :record_id, - entries: :all, - sort: [:index_tag, :varfield_type_code, :occurrence, :id] - }, - { - view: :varfield, - view_match: :record_id, obj_match: :record_id, - entries: :all, - sort: [:marc_tag, :varfield_type_code, :occ_num, :id] - }, - ] - - views.each do |hsh| - match_view(hsh) - access_view(hsh) - end - - def record_metadata - @record_metadata ||= read_record_metadata - end - - def self.record_metadata_struct - @record_metadata_struct ||= SierraDB.viewstruct(:record_metadata) - end - - statement = <<~SQL - select record_id - from sierra_view.phrase_entry phe - where (phe.index_tag || phe.index_entry) ~ $1::text - SQL - SierraDB.prepare_query('search_phrase_entry', statement) - - - statement = <<~SQL - select * - from sierra_view.record_metadata rm - where id = $1::bigint - SQL - SierraDB.prepare_query("id_find_record_metadata", statement) - - statement = <<~SQL - select * - from sierra_view.record_metadata rm - where record_type_code = $1::char - and record_num = $2::int - SQL - SierraDB.prepare_query("recnum_find_record_metadata", statement) - - # Reads/sets rec data from record_metadata by recnum lookup - def read_record_metadata - return {} unless recnum - - metadata = SierraDB.conn.exec_prepared( - 'recnum_find_record_metadata', - [rtype, recnum] - ).first&.values - return {} unless metadata - - SierraPostgresUtilities::Views::Record.record_metadata_struct.new( - *metadata - ) - end - end - end -end diff --git a/lib/sierra_postgres_utilities/views/user.rb b/lib/sierra_postgres_utilities/views/user.rb deleted file mode 100644 index 405be88..0000000 --- a/lib/sierra_postgres_utilities/views/user.rb +++ /dev/null @@ -1,35 +0,0 @@ -module SierraPostgresUtilities - module Views - module User - extend Views::MethodConstructor - - views = [ - { - view: :iii_user, - view_match: :name, obj_match: :login, - entries: :first, cast: :text - }, - { - view: :iii_user_permission_myuser, - view_match: :iii_user_id, obj_match: :id, - entries: :all, sort: :permission_num - }, - { - view: :iii_user_workflow, - view_match: :iii_user_id, obj_match: :id, - entries: :all, sort: :display_order - }, - { - view: :statistic_group_myuser, - view_match: :code, obj_match: :statistic_group_code_num, - entries: :first - }, - ] - - views.each do |hsh| - match_view(hsh) - access_view(hsh) - end - end - end -end diff --git a/sierra_postgres_utilities.gemspec b/sierra_postgres_utilities.gemspec index 73e040b..db52a81 100644 --- a/sierra_postgres_utilities.gemspec +++ b/sierra_postgres_utilities.gemspec @@ -1,17 +1,18 @@ # coding: utf-8 + lib = File.expand_path('../lib', __FILE__) $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) require 'sierra_postgres_utilities/version' Gem::Specification.new do |spec| spec.name = 'sierra_postgres_utilities' - spec.version = SierraPostgresUtilities::VERSION + spec.version = Sierra::VERSION spec.authors = ['ldss-jm', 'Kristina Spurgin'] spec.email = ['ldss-jm@users.noreply.github.com'] spec.summary = 'Connects to iii Sierra postgres DB and provides ' \ 'logic/utilities to interact with Sierra records in ruby.' - spec.homepage = "https://github.com/UNC-Libraries/sierra-postgres-utilities" + spec.homepage = 'https://github.com/UNC-Libraries/sierra-postgres-utilities' # Prevent pushing this gem to RubyGems.org. To allow pushes either set the 'allowed_push_host' # to allow pushing to a single host or delete this section to allow pushing to any host. @@ -22,18 +23,21 @@ Gem::Specification.new do |spec| 'public gem pushes.' end - spec.files = `git ls-files -z`.split("\x0").reject do |f| + spec.files = `git ls-files -z`.split("\x0").reject do |f| f.match(%r{^(test|spec|features)/}) end - spec.bindir = "exe" + spec.bindir = 'exe' spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) } - spec.require_paths = ["lib"] + spec.require_paths = ['lib'] spec.add_development_dependency 'bundler', '~> 1.14' spec.add_development_dependency 'rake', '~> 10.0' spec.add_development_dependency 'rspec', '~> 3.0' + spec.add_development_dependency 'factory_bot', '~> 5.0.0' spec.add_runtime_dependency 'mail', '~> 2.6' spec.add_runtime_dependency 'marc', '~> 1.0' spec.add_runtime_dependency 'pg', '~> 1.1' + spec.add_runtime_dependency 'sequel', '~> 5.20.0' + spec.add_runtime_dependency 'i18n', '~> 1.6.0' end diff --git a/spec/data/fields/control_field_spec.rb b/spec/data/fields/control_field_spec.rb new file mode 100644 index 0000000..9714d9e --- /dev/null +++ b/spec/data/fields/control_field_spec.rb @@ -0,0 +1,57 @@ +require 'spec_helper' + +describe Sierra::Data::ControlField do + let(:c006) { build(:control_006) } + let(:c007) { build(:control_007) } + let(:c008) { build(:control_008) } + + # For control fields, Sierra database stores a space for what should be a + # null position. (For example, p39 is meaningless/invalid for an 006, but + # the database will has p39 of ' ' rather than null.) + # 006s/008s are fixed length and we can assume any trailing spaces inside + # that length are part of the field + # 007s are variable length. We cannot trivially identify which trailing + # spaces are part of the actual 007 vs which are just part of the sierra db + # record, so we don't try to determine how many trailing spaces an 007 ought + # to have. + + describe '#to_s' do + context 'when an 006' do + it 'takes the first 18 characters' do + expect(c006.to_s.length).to eq(18) + end + + it 'does NOT strip trailing spaces' do + expect(c006.to_s).to eq('m u f ') + end + end + + context 'when an 007' do + it 'DOES strip trailing spaces' do + expect(c007.to_s).to eq('cr una|||unuua') + end + end + + context 'when an 008' do + it 'takes the first 40 characters' do + expect(c008.to_s.length).to eq(40) + end + + it 'does NOT strip trailing spaces' do + expect(c008.to_s).to eq('140912n| azannaabn |n aaa ') + end + end + end + + describe '#to_marc' do + it 'creates a MARC::ControlField object' do + expect(c006.to_marc).to be_a(MARC::ControlField) + end + + it 'uses #to_s for the ControlField value' do + expect(c006.to_marc.value).to eq(c006.to_s) + expect(c007.to_marc.value).to eq(c007.to_s) + expect(c008.to_marc.value).to eq(c008.to_s) + end + end +end diff --git a/spec/data/fields/leaderfield_spec.rb b/spec/data/fields/leaderfield_spec.rb new file mode 100644 index 0000000..ad18c66 --- /dev/null +++ b/spec/data/fields/leaderfield_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' + +describe Sierra::Data::LeaderField do + let(:ldr) { build(:leader) } + + describe '#to_s' do + subject { ldr.to_s } + + it 'returns leader field as a string' do + expect(subject).to eq('00000cam 2200145Ia 4500') + end + + it 'is 24 bytes/chars' do + expect(subject.length).to eq(24) + end + + it 'uses dummy values for record length (ldr/00-04)' do + expect(subject[0..4]).to eq('00000') + end + + it 'uses dummy values for indicator count (ldr/10)' do + expect(subject[10]).to eq('2') + end + + it 'uses dummy values for subfield code count (ldr/11)' do + expect(subject[11]).to eq('2') + end + + it 'uses dummy values for misc tail values (ldr/20-23)' do + expect(subject[20..23]).to eq('4500') + end + end +end diff --git a/spec/data/fields/varfield_spec.rb b/spec/data/fields/varfield_spec.rb new file mode 100644 index 0000000..0253137 --- /dev/null +++ b/spec/data/fields/varfield_spec.rb @@ -0,0 +1,150 @@ +require 'spec_helper' + +describe Sierra::Data::Varfield do + let(:v) { build(:varfield_marc) } + let(:v005) { build(:varfield_005) } + let(:v245) { build(:varfield_245) } + + context 'when a marc_varfield' do + describe '#marc_varfield?' do + it 'returns true' do + v.marc_tag = '100' + expect(v.marc_varfield?).to be true + end + end + + describe '#nonmarc_varfield?' do + it 'returns false' do + v.marc_tag = '100' + expect(v.nonmarc_varfield?).to be false + end + end + end + + context 'when NOT a marc_varfield' do + describe '#marc_varfield?' do + it 'returns false' do + v.marc_tag = nil + expect(v.marc_varfield?).to be false + end + end + + describe '#nonmarc_varfield?' do + it 'returns true' do + v.marc_tag = nil + expect(v.nonmarc_varfield?).to be true + end + end + + describe '#control_field?' do + context 'when marc_tag <= "009"' do + it 'returns true' do + expect(v005.control_field?).to be true + end + end + + context 'when marc_tag >= "010"' do + it 'returns false' do + v.marc_tag = '010' + expect(v.control_field?).to be false + end + end + end + + describe '#to_marc' do + context 'when varfield is a non_marc varfield' do + it 'returns nil' do + v.marc_tag = nil + expect(v.to_marc).to be_nil + end + end + + context 'when varfield is a control field' do + it 'returns a MARC::ControlField' do + expect(v005.to_marc).to eq( + MARC::ControlField.new('005', '19820807000000.0') + ) + end + end + + context 'when varfield is a data field' do + it 'returns a MARC::DataField' do + expect(v245.to_marc).to eq( + MARC::DataField.new('245', '1', '0', + ['a', 'Something else :'], + ['b', 'a novel']) + ) + end + end + end + + describe '.subfield_arry' do + subject { Sierra::Data::Varfield } + + let(:fc) { '|aIDEBK|beng|erda|cIDEBK|dCOO|aAAA' } + let(:fc_arry) do + [['a', 'IDEBK'], ['b', 'eng'], ['e', 'rda'], + ['c', 'IDEBK'], ['d', 'COO'], ['a', 'AAA']] + end + let(:fc_no_a) { 'IDEBK|beng|erda|cIDEBK|dCOO|aAAA' } + + it 'returns array of subfield,value pairs' do + expect(subject.subfield_arry(fc)).to eq(fc_arry) + end + + context 'when implicit_sfa: true' do + it 'adds initial |a if content lacks initial sf delimiter' do + expect(subject.subfield_arry(fc_no_a, implicit_sfa: true)). + to eq(fc_arry) + end + end + + it 'it implict_sfa is true by default' do + expect(subject.subfield_arry(fc_no_a)). + to eq(subject.subfield_arry(fc, implicit_sfa: true)) + end + + context 'when implicit_sfa: false' do + it 'does not add initial |a when content lacks initial sf delimiter' do + expect(subject.subfield_arry(fc_no_a, implicit_sfa: false)). + to eq(fc_arry[1..-1]) + end + end + + context 'when subfield lacks subfield code' do + it 'discards that subfield' do + expect(subject.subfield_arry('|adata||balso data')). + to eq([['a', 'data'], ['b', 'also data']]) + end + + it 'returns empty array if no subfields left' do + expect(subject.subfield_arry('|')).to eq([]) + end + end + end + + describe '.add_explicit_sf_a' do + subject { Sierra::Data::Varfield } + + let(:fc) { '|aIDEBK|beng|erda|cIDEBK|dCOO|aAAA' } + let(:fc_no_a) { 'IDEBK|beng|erda|cIDEBK|dCOO|aAAA' } + + context 'when field_content lacks initial "|a"' do + it 'adds explcit initial |a' do + expect(subject.add_explicit_sf_a(fc_no_a)).to eq(fc) + end + + it 'does not modify original object' do + subject.add_explicit_sf_a(fc_no_a) + expect(fc_no_a).to eq('IDEBK|beng|erda|cIDEBK|dCOO|aAAA') + end + end + + context 'when field_content lacks initial "|a"' do + it 'makes no changes' do + expect(subject.add_explicit_sf_a(fc)).to eq(fc) + end + end + end + end +end diff --git a/spec/data/helpers/phrase_normalization_spec.rb b/spec/data/helpers/phrase_normalization_spec.rb new file mode 100644 index 0000000..051ce7e --- /dev/null +++ b/spec/data/helpers/phrase_normalization_spec.rb @@ -0,0 +1,255 @@ +require 'spec_helper' + +module Sierra + module Data + module Helpers + RSpec.describe PhraseNormalization do + describe '.standard_normalize' do + pairs = [ + ['5916808|%1762189', ' 5916808 1762189'], + ['nchg2df651fe-7079-47a2-b29d-77ea90702dc1', + 'nchg2df651fe 7079 47a2 b29d 77ea90702dc1'], + ['|z60618545', '60618545'], + [' ', nil], + ['32988245|z(OCoLC)62441777|z(OCoLC)77633390|z(OCoLC)77633393' \ + '|z(OCoLC)77664829|z(OCoLC)77664832|z(OCoLC)77734146|z(OCoLC)' \ + '77768001|z(OCoLC)77819922|z(OCoLC)77867473|z(OCoLC)77879873' \ + '|z(OCoLC)77948921|z(OCoLC)77992118|z(OCoLC)78156448|z(OCoLC)' \ + '78202144|z(OCoLC)78227074|z(OCoLC)78229478|z(OCoLC)78263835', + '32988245 ocolc 62441777 ocolc 77633390 ocolc 77633393 ocolc 776' \ + '64829 ocolc 77664832 ocolc 77734146 ocolc 77768001 ocolc 77819'], + ['1,000,000 dollar dream', ' 1000000 dollar dream'], + ['1,000 best movies on DVD', ' 1000 best movies on dvd'], + ['$5,000 note $5,000', '$5000 note $5000'], + ['$500,000.00 real estate mortgage gold bonds : secured by the ' \ + 'holdings of the Winter Garden Estates, Asheville, North ' \ + 'Carolina ...', + '$500000 00 real estate mortgage gold bonds secured by the' \ + ' holdings of the winter garden estates asheville north carolin'], + ['100.000 candeles', ' 100 000 candeles'] + ] + + pairs.each do |orig, norm| + it "normalizes orig: #{orig} to norm: #{norm}" do + expect(PhraseNormalization.standard_normalize(orig)).to eq(norm) + end + end + end + + describe '.pad_numbers' do + it 'pads things' do + expect(PhraseNormalization.pad_numbers('2')).to eq(' 2') + end + + pairs = [ + ['B-1565-11', 'B- 1565-11'], + ['B-1565--11', 'B- 1565-- 11'], + ['B-1565-1-1', 'B- 1565- 1- 1'], + ['B-1565-111', 'B- 1565- 111'], + ['B-1565-11-11', 'B- 1565-11- 11'], + ['B-1565-a11a', 'B- 1565-a11a'], + ['nchg2df651fe-7079-47a2-b29d-77ea90702dc1', + 'nchg2df651fe- 7079-47a2-b29d- 77ea90702dc1'] + ] + pairs.each do |orig, norm| + it "normalizes orig: #{orig} to norm: #{norm}" do + expect(PhraseNormalization.pad_numbers(orig)).to eq(norm) + end + end + end + + describe '.number_normalize' do + isxns = [ + ['147800018X', '147800018x'], + ['978 11 11 11', '978111111'], + ['978 aa11 11 11', '978aa111111'], + ['0252-8169', '02528169'] + ] + isxns.each do |orig, norm| + it "normalizes isxns. orig: #{orig} to norm: #{norm}" do + expect(PhraseNormalization.number_normalize(orig)).to eq(norm) + end + end + + barcodes = [ + ['00001254305', '00001254305'], + ['0000 aaa 305', '0000aaa305'], + ['PL00014362', 'pl00014362'], + ['HAAA-9621-00003', 'haaa962100003'], + ['H00139989$', 'h00139989'] + ] + barcodes.each do |orig, norm| + it "normalizes barcodes. orig: #{orig} to norm: #{norm}" do + expect(PhraseNormalization.number_normalize(orig)).to eq(norm) + end + end + end + + describe '.sudoc_normalize' do + pairs = [ + ['Y 4.2:B 43/5/D 67/V.1-3', + 'y 4. 2 :b 43/ 5/d 67/v. 1- 3'], + ['Y4.2: B43/5/D67/v.3', + 'y 4. 2 :b 43/ 5/d 67/v. 3'], + ['1. 14/ 2 :f 31/ 6/ 2017', + ' 1. 14/ 2 :f 31/ 6/ 2017'], + ['NAS 1.26:189082', + 'nas 1. 26 :189082'], + ['E 1.86/14:', + 'e 1. 86/ 14 :'], + ['Ju 13 .9 :85-2', + 'ju 13. 9 : 85- 2'], + ['I 49.107:89 (2.21)', + 'i 49. 107 : 89( 2. 21)'], + ['Y4.SM 1: 115-047', + 'y 4.sm 1 : 115- 047'], + ['Y 4.L 11/4:S.HRG. 107-186', + 'y 4.l 11/ 4 :s.hrg. 107- 186'], + ['PRVP 42 .2 :G 74 /HUMAN', + 'prvp 42. 2 :g 74/human'], + ['NAS 1.15:2017- 219071', + 'nas 1. 15 : 2017-219071'], + ['FEM 1.209:300119, 300122, 300155', + 'fem 1. 209 :300119,300122,300155'], + ['D 101.11:5-2330-361-14 & P/984', + 'd 101. 11 : 5- 2330- 361- 14&p/ 984'], + ['D 45/7/Maps 3 & 4/2017', + 'd 45/ 7/maps 3& 4/ 2017'], + ['A 13.28:Su 7/7/47091-C 3 - C 8/photo.', + 'a 13. 28 :su 7/ 7/47091-c 3-c 8/photo.'], + ['GP 3.35:DOSṮERR', + 'gp 3. 35 :dosterr'], + ['FEM 1.209:260537', + 'fem 1. 209 :260537'], + ['GP 3.35:FAC3̲1FR', + 'gp 3. 35 :fac 31fr'], + ['J 28 .24/3 :Iℓ 5', + 'j 28. 24/ 3 :il 5'], + [' 1.14/2: F 31/6/2017', + ' 1. 14/ 2 :f 31/ 6/ 2017'], + ['ḎḆṈṞṮḺḆ', + 'dbnrtlb'] + ] + + pairs.each do |orig, norm| + it "normalizes sudocs. orig: #{orig} to norm: #{norm}" do + expect(PhraseNormalization.sudoc_normalize(orig)).to eq(norm) + end + end + end + + describe '.dewey_normalize' do + pairs = [ + ['373.757 M13e', ' 373.757 m13e'], + ['65-DVD18989', ' 65 dvd18989'], + ['CD-10,928', 'cd 10 928'], + ['CD,16,707', 'cd 16 707'], + ['1-2586 reel 13416, no. 06', + ' 1 2586 reel 13416 no. 06'], + ['1-2586 reel 12619 , no. 01', + ' 1 2586 reel 12619 no. 01'], + ["Sotheby's 1986.05.19", 'sothebys 1986.05.19'], + ['NCME #587', 'ncme 587'], + ['INLS242 Book#2', 'inls242 book 2'], + ['942.007 $B G14e', ' 942.007 b g14e'], + ['1-2586 reel 970+971, no. 12,1', + ' 1 2586 reel 970 971 no. 12 1'], + ['J Gutieârrez 1947a', 'j gutiearrez 1947a'], + ['1-662 Ser. QQ & ZZ Guide', + ' 1 662 ser. qq and zz guide'], + ["Sotheby's 2003.01.16-17,19", + 'sothebys 2003.01.16 17 19'], + ['1-2586 reel 1409, no. 14; reel no. 1410,', + ' 1 2586 reel 1409 no. 14 reel no. 1410'], + ['823 S43,1898 v.19', ' 823 s43 1898 v.19'], + ['Serial 1-30 reel 151-152,1120-1121', + 'serial 1 30 reel 151 152 1120 1121'], + ['Serial 1-30 reel 1497,1874-1875', + 'serial 1 30 reel 1497 1874 1875'], + ['CpX D257', 'cpx d257'], + ['832 S33, 1948', ' 832 s33 1948'], + ['Y 1 .4/1:', 'y 1 .4 1'], + ['1-2586 reel 1761,no. 05', + ' 1 2586 reel 1761 no. 05'], + ['426 Series 1.4,1.5', ' 426 series 1.4 1.5'], + ['822 F915歔o', ' 822 f915{22444c}o'] + ] + + pairs.each do |orig, norm| + it "normalizes dewey/local nos. orig: #{orig} to norm: #{norm}" do + expect(PhraseNormalization.dewey_normalize(orig)).to eq(norm) + end + end + + failing_pairs = [ + ['J838 Kr 252 ss', 'j838 kr{7F00FC}ss'], + ['FC-1̲5093', 'fc 15093'] + ] + + # fringe-y cases that don't currently succeed + failing_pairs.each do |orig, norm| + xit "normalizes dewey/local nos. orig: #{orig} to norm: #{norm}" do + expect(PhraseNormalization.dewey_normalize(orig)).to eq(norm) + end + end + end + + describe '.lc_normalize' do + pairs = [ + ['HC111.A1 E25', 'hc 111 a1 e25'], + ['CJ2666 .H68 1979', 'cj 2666 h68 1979'], + ['TX715.2.S68 F677 2011', 'tx 715.2 s68 f677 2011'], + ['QA3 .A57 no. 754', 'qa 3 a57 no 754'], + ['PQ2489 1927 v. 43', 'pq 2489 1927 v 43'], + ['M287 .P54 op.8 P3', 'm 287 p54 op8 p3'], + ['BR55 .M5 v.9, no.7', 'br 55 m5 v 9 no 7'], + ['K46 .N5 no. 5240-5241', 'k 46 n5 no 5240 5241'], + ['F2281.C32 V34 2008', 'f 2281 c32 v34 2008'], + ['CU-0072', 'cu 0072'], + ['KF49 .L44 no. P.L. 94-145', 'kf 49 l44 no pl 94 145'], + ['W 4i Z96 1950 no. 23', 'w 4 i z96 1950 no 23'], + ['R-12961', 'r 12961'], + ["R706 .T64 1726 no. 6 Superv'd", + 'r 706 t64 1726 no 6 supervd'], + ['WP/05/59', 'wp 05 59'], + ['Pamphlet 147', 'pamphlet 147'], + ['NCME #820', 'ncme 820'], + ['PQ153 .M69 no 26', 'pq 153 m69 no 26'], + ['Shelved as: Circulation ; v. 83, suppl. 4', + 'shelved as circulation v. 83 suppl. 4'], + ['PQ6001 .R47 año 34 1968', 'pq 6001 r47 ano 34 1968'], + ['HC445.5.Z7 K392+', 'hc 445.5 z7 k392'], + ['WT 30 M489s Sect. 1.1', 'wt 30 m489 s sect 1 1'], + ['PG5003 .N692 sv. 1', 'pg 5003 n692 sv 1'], + ['QK827 #b.J62', 'qk 827 bj62'], + ['GC512.N8 A43 no., 90-09', 'gc 512 n8 a43 no 90 09'], + ['Shelved as: Advances in experimental medicine and biology ;' \ + ' v. 322', + 'shelved as advances in experimental medicine and biology v. 322'] + ] + + pairs.each do |orig, norm| + it "normalizes lc call nos. orig: #{orig} to norm: #{norm}" do + expect(PhraseNormalization.lc_normalize(orig)).to eq(norm) + end + end + + failing_pairs = [ + ['2002-09', '2002 09'], + ['PQ8497.A65 &b Z74', 'pq 8497 a65 and b z74'], + ['PQ6254 .N69 Año. 2 no. 24', 'pq 6254 n69 ano 2 no 24'], + ['QD461 .S92v. 108, etc.', 'qd 461 s92 v 108 etc'], + ['PL2543 ǂb .Z467', 'pl 2543 {7 F01 C2 }b z467'] + ] + + # Somewhat fringe-y cases that don't currently succeed + failing_pairs.each do |orig, norm| + xit "normalizes lc call nos. orig: #{orig} to norm: #{norm}" do + expect(PhraseNormalization.lc_normalize(orig)).to eq(norm) + end + end + end + end + end + end +end diff --git a/spec/data/helpers/sierra_marc_spec.rb b/spec/data/helpers/sierra_marc_spec.rb new file mode 100644 index 0000000..110dc6c --- /dev/null +++ b/spec/data/helpers/sierra_marc_spec.rb @@ -0,0 +1,44 @@ +require 'spec_helper' + +module Sierra + module Data + module Helpers + RSpec.describe SierraMARC do + describe '#marc' do + xit 'tests pending' do + end + end + + describe '.compile_marc' do + xit 'factory tests pending' do + end + + context 'marc production' do + let(:bib) { Sierra::Record.get('b1841152a') } + let(:correct_mrc) do + MARC::Reader.new('spec/spec_data/b1841152a.mrc').to_a.first + end + + describe '#marc' do + it 'returns a MARC::Record object' do + expect(bib.marc).to be_a(MARC::Record) + end + + it 'contains correct marc fields' do + expect(bib.marc.fields).to eq(correct_mrc.fields) + end + + it 'returns proper leader, apart from dummied fields/chars' do + bib.marc.leader[0..4] = '00000' + bib.marc.leader[12..16] = '00000' + correct_mrc.leader[0..4] = '00000' + correct_mrc.leader[12..16] = '00000' + expect(bib.marc.leader).to eq(correct_mrc.leader) + end + end + end + end + end + end + end +end diff --git a/spec/data/properties/varfield_type_spec.rb b/spec/data/properties/varfield_type_spec.rb new file mode 100644 index 0000000..1a134cf --- /dev/null +++ b/spec/data/properties/varfield_type_spec.rb @@ -0,0 +1,15 @@ +require 'spec_helper' + +describe Sierra::Data::VarfieldType do + subject { Sierra::Data::VarfieldType } + + describe '.list' do + it 'returns a hash of vf code => vf_name for given record type' do + expect(subject.list('b')['b']).to eq('Added Author') + end + + it 'returns item vf code' do + expect(subject.list('i')['b']).to eq('Barcode') + end + end +end diff --git a/spec/data/record/authority_spec.rb b/spec/data/record/authority_spec.rb new file mode 100644 index 0000000..4474bd7 --- /dev/null +++ b/spec/data/record/authority_spec.rb @@ -0,0 +1,7 @@ +require 'spec_helper' + +describe Sierra::Data::Authority do + let(:metadata) { build(:metadata_a) } + let(:data) { build(:data_a) } + let(:auth) { newrec(Sierra::Data::Authority, metadata, data) } +end diff --git a/spec/data/record/bib_spec.rb b/spec/data/record/bib_spec.rb new file mode 100644 index 0000000..2f91699 --- /dev/null +++ b/spec/data/record/bib_spec.rb @@ -0,0 +1,165 @@ +require 'spec_helper' + +describe Sierra::Data::Bib do + let(:metadata) { build(:metadata_b) } + let(:data) { build(:data_b) } + let(:bib) { newrec(Sierra::Data::Bib, metadata, data) } + + let(:property) do + {id: 1076136, + bib_record_id: 420908636160, + best_title: 'Something else : a novel', + bib_level_code: 'm', + material_code: 'a', + publish_year: 1981, + best_title_norm: 'something else a novel', + best_author: 'Fassnidge, Virginia.', + best_author_norm: 'fassnidge virginia'} + end + + describe '#bnum' do + it 'returns rnum (including leading-letter and trailing-a)' do + expect(bib.bnum).to eq(bib.rnum) + expect(bib.bnum).to eq('b2661010a') + end + end + + describe '#bnum_trunc' do + it 'yields rnum without check digit or "a"' do + expect(bib.bnum_trunc).to eq('b2661010') + end + end + + describe '#bnum_with_check' do + it 'yields rnum including actual check digit' do + expect(bib.bnum_with_check).to eq('b26610103') + end + end + + context 'bib_record aliases' do + describe '#bcode1' do + it 'returns bcode1 from bib_record' do + expect(bib.bcode1).to eq('s') + end + end + + describe '#mat_type' do + it 'returns material type from bib_record_property' do + bib.set_data(:property, property) + expect(bib.mat_type).to eq('a') + end + end + end + + context 'leader aliases' do + subject { bib.set_data(:leader_field, build(:leader)) } + + describe '#rec_type' do + it 'returns record type code from leader' do + expect(subject.rec_type).to eq('a') + end + end + + describe '#blvl' do + it 'returns bib level code from leader' do + expect(subject.blvl).to eq('m') + end + end + + describe '#ctrl_type' do + it 'returns control type code from leader' do + expect(subject.ctrl_type).to eq(' ') + end + end + end + + describe '#location_codes' do + subject do + bib.set_data(:locations, [build(:loc_dd), + build(:loc_wb), + build(:loc_multi)]). + location_codes + end + + it 'returns array' do + expect(subject).to be_an(Array) + end + + it 'returns bib locations' do + expect(subject).to include('dd') + end + + it 'does not return other location' do + expect(subject).not_to include('ddda') + end + + it 'always excludes "multi"' do + expect(subject).not_to include('multi') + end + end + + describe '#best_title' do + it 'returns iii best_title' do + bib.set_data(:property, build(:bib_property)) + expect(bib.best_title).to eq('Something else : a novel') + end + end + + describe '#best_author' do + it 'returns iii best_author' do + bib.set_data(:property, build(:bib_property)) + expect(bib.best_author).to eq('Fassnidge, Virginia.') + end + end + + describe '#imprint' do + let(:bib) { Sierra::Record.get('b1841152a') } + + it 'returns cleaned value of first 260/264 field' do + expect(bib.imprint).to eq('London : Constable, 1981.') + end + + xit 'uses equivalent of extract_subfields' do + expect(bib.imprint).to eq('London : Constable, 1981.') + end + end + + describe '#items' do + it 'returns array of attached items' do + row = Sierra::DB.db[:bib_record_item_record_link]. + select(:bib_record_id, :item_record_id).first.values + b = Sierra::Record.get(id: row.first) + i = Sierra::Record.get(id: row.last) + expect(b.items).to include(i) + end + end + + describe '#holdings' do + it 'returns array of attached holdings records' do + row = Sierra::DB.db[:bib_record_holding_record_link]. + select(:bib_record_id, :holding_record_id).first.values + b = Sierra::Record.get(id: row.first) + h = Sierra::Record.get(id: row.last) + expect(b.holdings).to include(h) + end + end + + describe '#orders' do + it 'returns array of attached orders' do + row = Sierra::DB.db[:bib_record_order_record_link]. + select(:bib_record_id, :order_record_id).first.values + b = Sierra::Record.get(id: row.first) + o = Sierra::Record.get(id: row.last) + expect(b.orders).to include(o) + end + end + + describe '#oclcnum' do + it 'gets oclcnum from MARC::Record' do + marc = double('marc') + bib.instance_variable_set(:'@marc', marc) + allow(marc).to receive(:oclcnum).and_return('my_oclcnum') + expect(bib.oclcnum).to eq('my_oclcnum') + end + end +end diff --git a/spec/data/record/generic_spec.rb b/spec/data/record/generic_spec.rb new file mode 100644 index 0000000..ddf77fe --- /dev/null +++ b/spec/data/record/generic_spec.rb @@ -0,0 +1,131 @@ +require 'spec_helper' + +describe Sierra::Data::GenericRecord do + let(:metadata) { build(:metadata_a) } + let(:data) { build(:data_a) } + let(:rec) { newrec(Sierra::Data::Authority, metadata, data) } + + describe '#deleted?' do + context 'when record has been deleted' do + it 'returns true' do + rec2 = newrec(Sierra::Data::Authority, build(:metadata_deleted)) + expect(rec2.deleted?).to be true + end + end + + context 'when record has not been deleted' do + it 'returns false' do + expect(rec.deleted?).to be false + end + end + end + + describe '#suppressed?' do + context 'when record.is_suppressed' do + it 'returns true' do + rec.is_suppressed = true + expect(rec.suppressed?).to be true + end + end + + context 'when record.is_suppressed is false' do + it 'returns false' do + rec.is_suppressed = false + expect(rec.suppressed?).to be false + end + end + end + + describe '#record_id' do + it 'returns Sierra record id' do + expect(rec.record_id).to eq(416615865210) + end + end + + describe '#rnum' do + it 'returns Sierra rnum (including trailing a)' do + expect(rec.rnum).to eq('a2661010a') + end + end + + describe '#rnum_trunc' do + it 'returns Sierra rnum (omitting trailing a)' do + expect(rec.rnum_trunc).to eq('a2661010') + end + end + + describe '#rnum_with_check' do + it 'returns Sierra rnum including actual check digit' do + expect(rec.rnum_with_check).to eq('a26610103') + end + end + + describe '#recnum' do + it 'returns Sierra record_num (i.e. numbers only)' do + expect(rec.recnum).to eq('2661010') + end + end + + describe 'check_digit' do + it 'yields a string' do + expect(rec.check_digit('2661010')).to be_an(String) + end + + it 'calculates check digit for a recnum' do + expect(rec.check_digit('2661010')).to eq('3') + end + + it 'correctly calculates a check digit of "x"' do + expect(rec.check_digit('1191693')).to eq('x') + end + end + + describe '#standardize_rnum' do + xit 'tests pending' do + end + end + + describe '#type' do + xit 'tests pending' do + end + end + + describe '#created_date' do + subject { rec.created_date } + + it 'returns time created' do + expect(subject).to eq(Time.parse('2004-11-04 12:55:00 -0500')) + end + + it 'returns a Time object' do + expect(subject).to be_a(Time) + end + end + + describe '#updated_date' do + subject { rec.updated_date } + + it 'returns time last updated' do + expect(subject).to eq(Time.parse('2018-10-11 07:30:34 -0400')) + end + + it 'returns a Time object' do + expect(subject).to be_a(Time) + end + end + + describe '#varfields' do + xit 'tests pending' do + end + end + + describe '#vf_codes' do + it "returns a hash of vf code => vf_name for record's type" do + expect(Sierra::Data::Bib.first.vf_codes['b']).to eq('Added Author') + end + + it 'returns item vf code' do + expect(Sierra::Data::Item.first.vf_codes['b']).to eq('Barcode') + end + end +end diff --git a/spec/data/record/item_spec.rb b/spec/data/record/item_spec.rb new file mode 100644 index 0000000..36b0ade --- /dev/null +++ b/spec/data/record/item_spec.rb @@ -0,0 +1,213 @@ +require 'spec_helper' + +describe Sierra::Data::Item do + let(:metadata) { build(:metadata_i) } + let(:data) { build(:data_i) } + let(:item) { newrec(Sierra::Data::Item, metadata, data) } + + describe '#inum' do + it 'returns rnum (including leading-letter and trailing-a)' do + expect(item.inum).to eq(item.inum) + expect(item.inum).to eq('i2661010a') + end + end + + describe '#inum_trunc' do + it 'returns without check digit or "a"' do + expect(item.inum_trunc).to eq('i2661010') + end + end + + describe '#inum_with_check' do + it 'yields rnum including actual check digit' do + expect(item.inum_with_check).to eq('i26610103') + end + end + + describe 'helpers to access varfields by varfield type' do + context 'when varfields of that type exist' do + context 'by default (or when value_only is explicitly true)' do + it 'returns varfield(s) value/field_content as array of strings' do + item.set_data(:varfields, [build(:varfield_i_b)]) + expect(item.barcodes).to eq(['00050035567']) + end + end + + context 'when value_only is explicitly false' do + it 'returns varfield(s) as array of hash-like objects' do + item.set_data(:varfields, [build(:varfield_i_b)]) + expect(item.barcodes(value_only: false).first).to respond_to(:values) + end + end + end + + context 'otherwise' do + it 'is empty' do + item.set_data(:varfields, []) + expect(item.barcodes).to be_empty + end + end + end + + describe 'varfield retrieval by type' do + types = [ + {method: 'barcodes', values: ['00050035567']}, + {method: 'volumes', values: ['Suppl.']}, + {method: 'public_notes', values: ['Second nature ; Reflections']}, + {method: 'internal_notes', values: ['jc']}, + {method: 'messages', values: ['Message']}, + {method: 'stats_fields', values: ['VENDOR: YBP uncat']}, + {method: 'varfield_librarys', values: ['ART']}, + {method: 'callnos', values: ['TR655.H66 2015']} + ] + types.each do |type_hsh| + it "returns array of #{type_hsh[:method]} as strings" do + item.set_data( + :varfields, + [build(:varfield_i_b), build(:varfield_i_c), build(:varfield_i_f), + build(:varfield_i_j), build(:varfield_i_m), build(:varfield_i_v), + build(:varfield_i_x), build(:varfield_i_z)] + ) + expect(item.send(type_hsh[:method])).to eq(type_hsh[:values]) + end + end + end + + describe 'callnos' do + context 'by default (or when keep_delimiters: false)' do + it 'removes delimiters from field contents' do + item.set_data(:varfields, [build(:varfield_i_c)]) + expect(item.callnos.first).to eq('TR655.H66 2015') + end + end + context 'when keep_delimiters: true' do + it 'leaves delimiters in the field contents' do + item.set_data(:varfields, [build(:varfield_i_c)]) + expect(item.callnos(keep_delimiters: true).first).to eq( + '|aTR655|b.H66 2015' + ) + end + end + end + + describe '#icode2' do + it 'returns icode2' do + expect(item.icode2).to eq('-') + end + end + + describe '#itype_code' do + # note that item_record.itype_code_num is a number + it 'returns itype code as a string' do + expect(item.itype_code).to eq('0') + end + end + + describe '#location_code' do + it 'returns location code' do + expect(item.location_code).to eq('trln') + end + end + + describe '#status_code' do + it 'returns status code' do + expect(item.status_code).to eq('-') + end + end + + describe '#copy_num' do + it 'returns copy num as a number' do + expect(item.copy_num).to eq(1) + end + end + + describe '#checkout_total' do + it 'returns checkout_total as a number' do + expect(item.checkout_total).to eq(19) + end + end + + describe '#suppressed?' do + it 'returns boolean for suppression value' do + expect(item.suppressed?).to be false + end + end + + describe '#itype_desc' do + it 'returns itype description / longname' do + expect(item.itype_desc).to eq('Book') + end + end + + describe '#location_desc' do + it 'returns location description / longname' do + expect( + item.location_desc + ).to eq('Library Service Center — Request from Storage') + end + end + + describe '#status_desc' do + it 'returns status description / longname' do + expect(item.status_desc).to eq('Available') + end + end + + describe '#due_date' do + context 'when item is checked out' do + it 'returns due date' do + item.set_data(:checkout, build(:checkout)) + expect(item.due_date).to eq(Time.parse('2019-01-02 00:00:00 -0500')) + end + + it 'returns due date as Time object' do + item = Sierra::Data::Checkout.first.item + expect(item.due_date).to be_a(Time) + end + end + + context 'when item is not checked out' do + it 'returns nil' do + item.set_data(:checkout, nil) + expect(item.due_date).to be_nil + end + end + end + + describe '#bib' do + it 'returns array of attached bibs' do + row = Sierra::DB.db[:bib_record_item_record_link]. + select(:bib_record_id, :item_record_id).first.values + b = Sierra::Record.get(id: row.first) + i = Sierra::Record.get(id: row.last) + expect(i.bibs).to include(b) + end + end + + describe '#holdings' do + it 'returns attached holdings record' do + row = Sierra::DB.db[:holding_record_item_record_link]. + select(:holding_record_id, :item_record_id).first.values + h = Sierra::Record.get(id: row.first) + i = Sierra::Record.get(id: row.last) + expect(i.holdings).to eq(h) + end + end + + describe '#is_oca?' do + xit 'returns true when book note present' do + oca_book = Sierra::Record.get('i7364701a') + expect(oca_book.is_oca?).to be true + end + + xit 'returns true when journal note present' do + oca_journal = Sierra::Record.get('i7364813a') + expect(oca_journal.is_oca?).to be true + end + + xit 'falsey if no oca note present' do + non_oca = Sierra::Record.get('i1000035a') + expect(non_oca.is_oca?).to be_falsey + end + end +end diff --git a/spec/db/connection_spec.rb b/spec/db/connection_spec.rb new file mode 100644 index 0000000..0c45c1a --- /dev/null +++ b/spec/db/connection_spec.rb @@ -0,0 +1,14 @@ +require 'spec_helper' + +module Sierra + module DB + RSpec.describe Connection do + describe '.base_dir' do + it 'string path to the base sierra-postgres-utilities dir' do + expect(File.join(Connection.base_dir, 'spec', 'db', + 'connection_spec.rb')).to eq(__FILE__) + end + end + end + end +end diff --git a/spec/db/query_spec.rb b/spec/db/query_spec.rb new file mode 100644 index 0000000..278cc7b --- /dev/null +++ b/spec/db/query_spec.rb @@ -0,0 +1,17 @@ +require 'spec_helper' + +module Sierra + module DB + RSpec.describe Query do + let(:query) { 'select * from sierra_view.bib_record limit 1' } + let(:io) { StringIO.new } + before(:each) { Sierra::DB.query(query) } + + describe '.headers' do + it 'returns array of field names as symbols' do + expect(Sierra::DB::Query.headers.first).to eq(:id) + end + end + end + end +end diff --git a/spec/db_spec.rb b/spec/db_spec.rb new file mode 100644 index 0000000..ab0edf4 --- /dev/null +++ b/spec/db_spec.rb @@ -0,0 +1,57 @@ +require 'spec_helper' + +module Sierra + RSpec.describe DB do + let(:query) { 'select * from sierra_view.bib_record limit 1' } + let(:io) { StringIO.new } + before(:each) { Sierra::DB.query(query) } + + describe '.query' do + it 'executes/stages an arbitrary sql query' do + expect(Sierra::DB.query(query)).to be_a(Sequel::Dataset) + expect(Sierra::DB.query(query).sql).to eq(query) + end + end + + describe '.write_results' do + context 'with include_headers: true (default)' do + it 'includes headers' do + Sierra::DB.write_results(io) + expect(io.string[0..1]).to eq('id') + end + end + + context 'with include_headers: false' do + it 'omits headers' do + Sierra::DB.write_results(io, include_headers: false) + expect(io.string[0..1]).to match(/[0-9]*/) + end + end + + context 'with format: tsv (default)' do + it 'writes to a tsv' do + Sierra::DB.write_results(io) + expect(io.string.each_line.first.split("\t").first).to eq('id') + end + end + + context 'with format: csv (default)' do + it 'writes to a csv' do + Sierra::DB.write_results(io, format: :csv) + expect(io.string.each_line.first.split(',').first).to eq('id') + end + end + + context ' with format: xlsx (default)' do + xit 'writes to an xlsx' do + end + end + end + + describe '.mail_results' do + end + + describe '.yield_email' do + end + end +end diff --git a/spec/derivative_bib_spec.rb b/spec/derivative_bib_spec.rb new file mode 100644 index 0000000..d6e43ab --- /dev/null +++ b/spec/derivative_bib_spec.rb @@ -0,0 +1,80 @@ +require_relative '../lib/sierra_postgres_utilities.rb' + +RSpec.describe Sierra::DerivativeBib do + let(:marc) do + MARC::Reader.new('spec/spec_data/b1841152a.mrc').to_a.first + end + + let(:bib) do + b = Sierra::Record.get('b1841152a') + b.marc = marc + b + end + + let(:alt) { Sierra::DerivativeBib.new(bib) } + + let(:xml) do + <<~XML + + 00469cam 2200169Ia 4500 + b1841152 + NcU + 19820807000000.0 + 820807s1981 enk 000 1 eng d + + 0094643407 + + + (OCoLC)8671134 + + + NOC + NOC + + + Fassnidge, Virginia. + + + Something else : + a novel / + Virginia Fassnidge. + + + London : + Constable, + 1981. + + + 152 p. ; + 23 cm. + + + XML + end + + describe '#get_alt_marc' do + it 'modifies a copy of the marc on the sierra bib' do + alt.altmarc + expect(alt.smarc['001'].value).to eq('8671134') + end + + it 'writes bnum_trunc to 001' do + expect(alt.altmarc['001'].value).to eq(bib.bnum_trunc) + end + + it 'writes an 001 oclcnumber to the 035' do + expect(alt.altmarc['035'].value).to eq('(OCoLC)8671134') + end + end + + describe '#xml' do + it 'returns altmarc as xml' do + expect(alt.xml).to eq(File.read('spec/spec_data/b1841152a.altmarc.xml')) + end + + it 'accepts strip_datafields: false' do + marc << MARC::DataField.new('030', ' ', ' ', ['a', 'blah ']) + expect(alt.xml(strip_datafields: false)).to include('blah ') + end + end +end diff --git a/spec/derivative_record_spec.rb b/spec/derivative_record_spec.rb deleted file mode 100644 index 7b978f4..0000000 --- a/spec/derivative_record_spec.rb +++ /dev/null @@ -1,21 +0,0 @@ -require_relative '../lib/sierra_postgres_utilities.rb' - -RSpec.describe DerivativeRecord do - describe '#get_alt_marc' do - let(:bib) { SierraBib.new('b1841152a') } - let(:alt) { DerivativeRecord.new(bib) } - - it 'modifies a copy of the marc on the sierra bib' do - alt.altmarc - expect(alt.smarc['001'].value).to eq('8671134') - end - - it 'writes bnum_trunc to 001' do - expect(alt.altmarc['001'].value).to eq(bib.bnum_trunc) - end - - it 'writes an 001 oclcnumber to the 035' do - expect(alt.altmarc['035'].value).to eq('(OCoLC)8671134') - end - end -end diff --git a/spec/ext/marc/datafield_spec.rb b/spec/ext/marc/datafield_spec.rb index 8efd746..7281e30 100644 --- a/spec/ext/marc/datafield_spec.rb +++ b/spec/ext/marc/datafield_spec.rb @@ -1,14 +1,11 @@ -require 'marc' -require_relative '../../../ext/marc/datafield' - +require 'spec_helper' RSpec.describe MARC::DataField do - describe 'to_mrk' do - + describe '#to_mrk' do f = MARC::DataField.new('999', '1', '2', ['a', 'content']) it 'yields a mrk-type string of the field' do - expect(f.to_mrk).to eq ('=999 12$acontent') + expect(f.to_mrk).to eq('=999 12$acontent') end it 'uses dollar sign subfield delimiter (MarcEdit-style)' do @@ -26,13 +23,15 @@ fempty = MARC::DataField.new('999', '1', '2') it 'produces string even if no subfield data is present' do - expect(fempty.to_mrk).to eq ('=999 12') + expect(fempty.to_mrk).to eq('=999 12') end end - describe '#any_subfields_ignore_repeated?' do - let(:a300) { MARC::DataField.new('300', '1', ' ', ['a', 'foo'], ['a', 'bar'], ['k', 'baz']) } + let(:a300) do + MARC::DataField.new('300', '1', ' ', ['a', 'foo'], + ['a', 'bar'], ['k', 'baz']) + end it 'looks at non-repeated subfields and first instance of repeated subfields' do expect(a300.any_subfields_ignore_repeated?(code: 'a', value: 'foo')).to be true @@ -41,16 +40,16 @@ it 'ignores second, third, etc instances of repeated subfields' do expect(a300.any_subfields_ignore_repeated?(code: 'a', value: 'bar')).to be false end - - # it searches the first subfield for eac matching code + + # it searches the first subfield for each matching code # not searches the first subfield with a matching code it 'searches first instance of _each_ matching subfield code' do expect(a300.any_subfields_ignore_repeated?(code: /[ak]/, value: 'baz')).to be true end end - # True when the first subfield a contains foo - # first_such_subfield_matches?(code: 'a', content: /foo/) - # True when the first subfield a or first subfield k contains foo - # first_such_subfield_matches?(code: /[ak]/, content: /foo/) -end \ No newline at end of file + # True when the first subfield a contains foo + # first_such_subfield_matches?(code: 'a', content: /foo/) + # True when the first subfield a or first subfield k contains foo + # first_such_subfield_matches?(code: /[ak]/, content: /foo/) +end diff --git a/spec/ext/marc/record_spec.rb b/spec/ext/marc/record_spec.rb index 4366658..303f5a3 100644 --- a/spec/ext/marc/record_spec.rb +++ b/spec/ext/marc/record_spec.rb @@ -1,11 +1,14 @@ -require 'marc' -require_relative '../../../ext/marc/record' +require 'spec_helper' -def ocn_stub_builder(_001, _003, _035s) +def ocn_stub_builder(m001, m003, m035s) rec = MARC::Record.new - rec << MARC::ControlField.new('001', _001) if _001 - rec << MARC::ControlField.new('003', _003) if _003 - _035s.each { |v| rec << MARC::DataField.new('035', ' ', ' ', ['a', v]) } if _035s + rec << MARC::ControlField.new('001', m001) if m001 + rec << MARC::ControlField.new('003', m003) if m003 + if m035s + m035s.each do |v| + rec << MARC::DataField.new('035', ' ', ' ', ['a', v]) + end + end rec end @@ -47,10 +50,10 @@ def new_rec_with_fields(*fields) expect(r.get_oclcnum).to eq(nil) end -# it 'strips leading zero(s) from oclc_number set from 035' do -# r = ocn_stub_builder('123', 'ItFiC', ['(OCoLC)000000567']) -# expect(r.get_oclcnum).to eq('567') -# end + # it 'strips leading zero(s) from oclc_number set from 035' do + # r = ocn_stub_builder('123', 'ItFiC', ['(OCoLC)000000567']) + # expect(r.get_oclcnum).to eq('567') + # end it 'sets oclc_number when 001 is digits only and 003 = NhCcYBP' do r = ocn_stub_builder('123', 'NhCcYBP', []) @@ -72,10 +75,10 @@ def new_rec_with_fields(*fields) expect(r.get_oclcnum).to eq(nil) end -# it 'sets oclc_number from 035 when it starts with ocm' do -# r = ocn_stub_builder('M-ESTCN123', 'OCoLC', ['(OCoLC)M-ESTCN987', '(OCoLC)ocm444']) -# expect(r.get_oclcnum).to eq('444') -# end + # it 'sets oclc_number from 035 when it starts with ocm' do + # r = ocn_stub_builder('M-ESTCN123', 'OCoLC', ['(OCoLC)M-ESTCN987', '(OCoLC)ocm444']) + # expect(r.get_oclcnum).to eq('444') + # end it 'does NOT set oclc_number when 001 has prefix moml and 003 = OCoLC' do r = ocn_stub_builder('moml123', 'OCoLC', []) @@ -94,7 +97,6 @@ def new_rec_with_fields(*fields) end describe 'get_035oclcnums' do - it 'returns oclc_number even when 001 is digits only and 003 = OCoLC' do r = ocn_stub_builder('123', 'OCoLC', ['(OCoLC)000000567']) expect(r.get_035oclcnums).to eq(['567']) @@ -130,7 +132,6 @@ def new_rec_with_fields(*fields) r = ocn_stub_builder('123', 'ItFiC', ['(OCoLC)']) expect(r.get_035oclcnums).to be_nil end - end describe 'oclcnum' do @@ -145,21 +146,50 @@ def new_rec_with_fields(*fields) end end + describe '.xml_string' do + let(:marc) do + MARC::Reader.new('spec/spec_data/b1841152a.mrc').to_a.first + end + + it 'returns given marc as xml' do + expect(MARC::Record.xml_string(marc)). + to eq(File.read('spec/spec_data/b1841152a.xml')) + end + + context 'when strip_datafields: true' do + it 'strips trailing spaces from data' do + marc << MARC::DataField.new('030', ' ', ' ', ['a', 'blah ']) + expect(MARC::Record.xml_string(marc, strip_datafields: true)).not_to include('blah ') + end + + it 'and strip_datafields: true is the default' do + marc << MARC::DataField.new('030', ' ', ' ', ['a', 'blah ']) + expect(MARC::Record.xml_string(marc)).not_to include('blah ') + end + end + + context 'when strip_datafields: false' do + it 'does not strip trailing spaces from data' do + marc << MARC::DataField.new('030', ' ', ' ', ['a', 'blah ']) + expect(MARC::Record.xml_string(marc, strip_datafields: false)).to include('blah ') + end + end + end describe '.language_from_008' do - let(:eng008) { + let(:eng008) do MARC::ControlField.new('008', '0.........1.........2.........3....eng...') - } - let(:invalid008) { + end + let(:invalid008) do MARC::ControlField.new('008', '0.........1.........2.........3....xxx...') - } - let(:discontinued008) { + end + let(:discontinued008) do MARC::ControlField.new('008', '0.........1.........2.........3....cam...') - } + end let(:rec_eng) { new_rec_with_fields(eng008) } let(:rec_invalid) { new_rec_with_fields(invalid008) } let(:rec_discontinued) { new_rec_with_fields(discontinued008) } - let(:rec_no008) { new_rec_with_fields() } + let(:rec_no008) { new_rec_with_fields } context 'when record has current, valid language code' do it 'returns language code, language name pair' do @@ -174,7 +204,6 @@ def new_rec_with_fields(*fields) end context 'when record has discontinued language code' do - context 'by default (or if forbid_discontinued is explicitly false)' do it 'returns language code, language name pair' do expect(rec_discontinued.language_from_008).to eq(['cam', 'Khmer']) @@ -188,8 +217,6 @@ def new_rec_with_fields(*fields) ).to eq(['cam', nil]) end end - - end context 'when record has no 008 / no 008/35-37' do @@ -225,9 +252,7 @@ def new_rec_with_fields(*fields) end end - describe 'm300_without_a' do - rec1 = MARC::Record.new rec1 << MARC::DataField.new('300', ' ', ' ', ['a', '']) rec1 << MARC::DataField.new('300', ' ', ' ', ['b', '']) @@ -246,7 +271,6 @@ def new_rec_with_fields(*fields) it 'is false if no 300s exist' do expect(rec4.m300_without_a?).to be false end - end describe 'count' do @@ -272,11 +296,11 @@ def new_rec_with_fields(*fields) let(:a999) { MARC::DataField.new('999', ' ', ' ', ['3', 'v.1']) } let(:a856) { MARC::DataField.new('856', ' ', ' ', ['3', 'v.2']) } let(:b856) { MARC::DataField.new('856', ' ', ' ', ['3', 'v.1']) } - let(:rec) { + let(:rec) do r = MARC::Record.new [a999, a856, b856].each { |f| r.append(f) } r - } + end it 'reorders fields to be in ascending order by tag' do expect(rec.sort.fields.last).to eq(a999) end @@ -284,23 +308,24 @@ def new_rec_with_fields(*fields) it 'within a field tag (e.g. 856) retains original order' do expect(rec.sort.fields.first).to eq(a856) end - end describe 'field_find_all' do let(:a300) { MARC::DataField.new('300', '1', ' ', ['a', 'content']) } let(:b300) { MARC::DataField.new('300', '1', '2', ['a', 'other']) } - let(:a900) { MARC::DataField.new( - '900', ' ', '1', ['a', 'content'], ['a', ' more content'] - )} - let(:b900) { MARC::DataField.new( - '900', ' ', '1', ['a', 'content'], ['b', ' more content'] - )} - let(:rec) { + let(:a900) do + MARC::DataField.new('900', ' ', '1', + ['a', 'content'], ['a', ' more content']) + end + let(:b900) do + MARC::DataField.new('900', ' ', '1', + ['a', 'content'], ['b', ' more content']) + end + let(:rec) do r = MARC::Record.new [a300, b300, a900, b900].each { |f| r.append(f) } r - } + end context 'when filtering by tag' do context 'and positive criteria specified' do @@ -423,28 +448,26 @@ def new_rec_with_fields(*fields) end context 'when complex_subfields specified' do - it 'requires complex_subfields to be an array of arrays' do - #todo + xit 'requires complex_subfields to be an array of arrays' do end it 'allows searching for fields having a matching subfield' do expect(rec.field_find_all( - complex_subfields: [[:has, code: 'a', value: ' more content']] - )).to eq([a900]) + complex_subfields: [[:has, code: 'a', value: ' more content']] + )).to eq([a900]) end it 'allows searching for fields lacking a matching subfield' do expect(rec.field_find_all( - complex_subfields: [[:has_no, code: 'a', value: /content/]] - )).to eq([b300]) + complex_subfields: [[:has_no, code: 'a', value: /content/]] + )).to eq([b300]) end it 'allows searching for fields having only one matching subfield' do expect(rec.field_find_all( - complex_subfields: [[:has_one, code_not: 'b']] - )).to eq([a300, b300, b900]) + complex_subfields: [[:has_one, code_not: 'b']] + )).to eq([a300, b300, b900]) end - end context 'when multiple critera are specified' do diff --git a/spec/ext/marc/xml_helper_spec.rb b/spec/ext/marc/xml_helper_spec.rb new file mode 100644 index 0000000..13734c0 --- /dev/null +++ b/spec/ext/marc/xml_helper_spec.rb @@ -0,0 +1,36 @@ +require 'spec_helper' + +module MARC + RSpec.describe XMLHelper do + describe '#escape_xml_reserved' do + xit 'passes data to XMLHelper.escape_xml_reserved and returns result' do + end + end + + describe '.escape_xml_reserved' do + it 'escapes ampersands' do + data = 'foo&bar' + expect(XMLHelper.escape_xml_reserved(data)). + to eq('foo&bar') + end + + it 'escapes left/right angle brackets' do + data = '' + expect(XMLHelper.escape_xml_reserved(data)). + to eq('<foo>') + end + + it 'escapes single quotes' do + data = "'foo'" + expect(XMLHelper.escape_xml_reserved(data)). + to eq(''foo'') + end + + it 'escapes double quotes' do + data = '"foo"' + expect(XMLHelper.escape_xml_reserved(data)). + to eq('"foo"') + end + end + end +end diff --git a/spec/ext/sequel/model/model_spec.rb b/spec/ext/sequel/model/model_spec.rb new file mode 100644 index 0000000..9d59645 --- /dev/null +++ b/spec/ext/sequel/model/model_spec.rb @@ -0,0 +1,45 @@ +require 'spec_helper' + +module Sequel + RSpec.describe Model do + describe '.prepare_retrieval_by' do + let(:user) { Sierra::Data::User.first } + Sierra::Data::User.prepare_retrieval_by(:name, :first) + Sierra::Data::User. + prepare_retrieval_by(:account_unit, :select, + sorting: %i[last_password_change_gmt name]) + + it 'creates a prepared statement' do + expect(Sierra::DB.db.prepared_statements.keys).to include(:user_by_name) + end + + it 'defines a "by_{field} method' do + expect(Sierra::Data::User.by_name(name: user.name)).to eq(user) + end + + context 'with a select_method of :first' do + it 'returns the first matching object' do + expect(Sierra::Data::User.by_name(name: user.name)).to eq(user) + end + end + + context 'with a select_method of :select' do + it 'returns an array of matching objects' do + expect(Sierra::Data::User. + by_account_unit(account_unit: user.account_unit)). + to include(user) + end + end + + it 'sorts results when provided sort order (at statement creation)' do + users = Sierra::Data::User. + by_account_unit(account_unit: user.account_unit) + sorted = users.sort_by do |u| + [u.last_password_change_gmt || Time.now, u.name] + end + + expect(users).to eq(sorted) + end + end + end +end diff --git a/spec/factories/authority.rb b/spec/factories/authority.rb new file mode 100644 index 0000000..5025548 --- /dev/null +++ b/spec/factories/authority.rb @@ -0,0 +1,15 @@ +module Sierra + module Data + FactoryBot.define do + factory :data_a, class: Authority do + id { 416615865210 } + record_id { 416615865210 } + marc_type_code { ' ' } + code1 { '-' } + code2 { '-' } + suppress_code { '-' } + is_suppressed { false } + end + end + end +end diff --git a/spec/factories/bib.rb b/spec/factories/bib.rb new file mode 100644 index 0000000..3139b96 --- /dev/null +++ b/spec/factories/bib.rb @@ -0,0 +1,23 @@ +module Sierra + module Data + FactoryBot.define do + factory :data_b, class: Bib do + id { 420907986691 } + record_id { 420907986691 } + language_code { 'eng' } + bcode1 { 's' } + bcode2 { 'a' } + bcode3 { '-' } + country_code { 'onc' } + index_change_count { 17 } + is_on_course_reserve { false } + is_right_result_exact { false } + allocation_rule_code { nil } + skip_num { 0 } + cataloging_date_gmt { Time.parse('2005-07-13 00:00:00 -0400') } + marc_type_code { ' ' } + is_suppressed { false } + end + end + end +end diff --git a/spec/factories/bib_record_property.rb b/spec/factories/bib_record_property.rb new file mode 100644 index 0000000..60b0fc9 --- /dev/null +++ b/spec/factories/bib_record_property.rb @@ -0,0 +1,17 @@ +module Sierra + module Data + FactoryBot.define do + factory :bib_property, class: BibRecordProperty do + id { 1076136 } + bib_record_id { 420908636160 } + best_title { 'Something else : a novel' } + bib_level_code { 'm' } + material_code { 'a' } + publish_year { 1981 } + best_title_norm { 'something else a novel' } + best_author { 'Fassnidge, Virginia.' } + best_author_norm { 'fassnidge virginia' } + end + end + end +end diff --git a/spec/factories/checkout.rb b/spec/factories/checkout.rb new file mode 100644 index 0000000..e5a1e9c --- /dev/null +++ b/spec/factories/checkout.rb @@ -0,0 +1,20 @@ +module Sierra + module Data + FactoryBot.define do + factory :checkout, class: Checkout do + id { 654949 } + patron_record_id { 400000000000 } + item_record_id { 450974227090 } + items_display_order { nil } + due_gmt { Time.parse('2019-01-02 00:00:00 -0500') } + loanrule_code_num { 1 } + checkout_gmt { Time.parse('2019-01-01 00:00:00 -0500') } + renewal_count { 0 } + overdue_count { 2 } + overdue_gmt { Time.parse('2019-01-03 00:00:00 -0500') } + recall_gmt { nil } + ptype { 0 } + end + end + end +end diff --git a/spec/factories/control_field.rb b/spec/factories/control_field.rb new file mode 100644 index 0000000..c46ae49 --- /dev/null +++ b/spec/factories/control_field.rb @@ -0,0 +1,159 @@ +module Sierra + module Data + FactoryBot.define do + factory :control, class: ControlField do + id { 111363035 } + record_id { 416615739221 } + varfield_type_code { 'y' } + occ_num { 3 } + + factory :control_006 do + control_num { 6 } + p00 { 'm' } + p01 { ' ' } + p02 { ' ' } + p03 { ' ' } + p04 { ' ' } + p05 { ' ' } + p06 { ' ' } + p07 { ' ' } + p08 { ' ' } + p09 { 'u' } + p10 { ' ' } + p11 { 'f' } + p12 { ' ' } + p13 { ' ' } + p14 { ' ' } + p15 { ' ' } + p16 { ' ' } + p17 { ' ' } + p18 { ' ' } + p19 { ' ' } + p20 { ' ' } + p21 { ' ' } + p22 { ' ' } + p23 { ' ' } + p24 { ' ' } + p25 { ' ' } + p26 { ' ' } + p27 { ' ' } + p28 { ' ' } + p29 { ' ' } + p30 { ' ' } + p31 { ' ' } + p32 { ' ' } + p33 { ' ' } + p34 { ' ' } + p35 { ' ' } + p36 { ' ' } + p37 { ' ' } + p38 { ' ' } + p39 { ' ' } + p40 { ' ' } + p41 { ' ' } + p42 { ' ' } + p43 { ' ' } + remainder { nil } + end + + factory :control_007 do + control_num { 7 } + p00 { 'c' } + p01 { 'r' } + p02 { ' ' } + p03 { 'u' } + p04 { 'n' } + p05 { 'a' } + p06 { '|' } + p07 { '|' } + p08 { '|' } + p09 { 'u' } + p10 { 'n' } + p11 { 'u' } + p12 { 'u' } + p13 { 'a' } + p14 { ' ' } + p15 { ' ' } + p16 { ' ' } + p17 { ' ' } + p18 { ' ' } + p19 { ' ' } + p20 { ' ' } + p21 { ' ' } + p22 { ' ' } + p23 { ' ' } + p24 { ' ' } + p25 { ' ' } + p26 { ' ' } + p27 { ' ' } + p28 { ' ' } + p29 { ' ' } + p30 { ' ' } + p31 { ' ' } + p32 { ' ' } + p33 { ' ' } + p34 { ' ' } + p35 { ' ' } + p36 { ' ' } + p37 { ' ' } + p38 { ' ' } + p39 { ' ' } + p40 { ' ' } + p41 { ' ' } + p42 { ' ' } + p43 { ' ' } + remainder { nil } + end + + factory :control_008 do + control_num { 8 } + p00 { '1' } + p01 { '4' } + p02 { '0' } + p03 { '9' } + p04 { '1' } + p05 { '2' } + p06 { 'n' } + p07 { '|' } + p08 { ' ' } + p09 { 'a' } + p10 { 'z' } + p11 { 'a' } + p12 { 'n' } + p13 { 'n' } + p14 { 'a' } + p15 { 'a' } + p16 { 'b' } + p17 { 'n' } + p18 { ' ' } + p19 { ' ' } + p20 { ' ' } + p21 { ' ' } + p22 { ' ' } + p23 { ' ' } + p24 { ' ' } + p25 { ' ' } + p26 { ' ' } + p27 { ' ' } + p28 { '|' } + p29 { 'n' } + p30 { ' ' } + p31 { 'a' } + p32 { 'a' } + p33 { 'a' } + p34 { ' ' } + p35 { ' ' } + p36 { ' ' } + p37 { ' ' } + p38 { ' ' } + p39 { ' ' } + p40 { 'n' } + p41 { 'z' } + p42 { ' ' } + p43 { 'n' } + remainder { ' ' } + end + end + end + end +end diff --git a/spec/factories/holdings.rb b/spec/factories/holdings.rb new file mode 100644 index 0000000..4eb75b6 --- /dev/null +++ b/spec/factories/holdings.rb @@ -0,0 +1,35 @@ +module Sierra + module Data + FactoryBot.define do + factory :data_c, class: Holdings do + id { 425211911992 } + record_type_code { 'c' } + record_num { 10149688 } + creation_date_gmt { Time.parse('2002-01-16 10:31:00 -0500') } + deletion_date_gmt { nil } + campus_code { '' } + agency_code_num { 0 } + num_revisions { 36 } + record_last_updated_gmt { Time.parse('2018-01-29 17:28:29 -0500') } + previous_last_updated_gmt { Time.parse('2017-08-02 16:16:00 -0400') } + record_id { 425211911992 } + is_inherit_loc { false } + allocation_rule_code { '0' } + accounting_unit_code_num { 2 } + label_code { 'n' } + scode1 { 'c' } + scode2 { '-' } + claimon_date_gmt { nil } + receiving_location_code { '1' } + vendor_code { 'yankf' } + scode3 { '-' } + scode4 { '-' } + update_cnt { 'i' } + piece_cnt { 0 } + echeckin_code { ' ' } + media_type_code { ' ' } + is_suppressed { false } + end + end + end +end diff --git a/spec/factories/item.rb b/spec/factories/item.rb new file mode 100644 index 0000000..79ea4f4 --- /dev/null +++ b/spec/factories/item.rb @@ -0,0 +1,45 @@ +module Sierra + module Data + FactoryBot.define do + factory :data_i, class: Item do + id { 450974227090 } + record_id { 450974227090 } + icode1 { 0 } + icode2 { '-' } + itype_code_num { 0 } + location_code { 'trln' } + agency_code_num { 0 } + item_status_code { '-' } + is_inherit_loc { false } + price { BigDecimal('0.0') } + last_checkin_gmt { Time.parse('2016-09-21 12:18:00 -0400') } + checkout_total { 19 } + renewal_total { 2 } + last_year_to_date_checkout_total { 1 } + year_to_date_checkout_total { 0 } + is_bib_hold { false } + copy_num { 1 } + checkout_statistic_group_code_num { 0 } + last_patron_record_metadata_id { 400000000000 } + inventory_gmt { nil } + checkin_statistics_group_code_num { 1 } + use3_count { 0 } + last_checkout_gmt { Time.parse('2016-09-21 12:17:00 -0400,') } + internal_use_count { 0 } + copy_use_count { 0 } + item_message_code { '-' } + opac_message_code { '-' } + virtual_type_code { nil } + virtual_item_central_code_num { 0 } + holdings_code { '6' } + save_itype_code_num { nil } + save_location_code { nil } + save_checkout_total { nil } + old_location_code { nil } + distance_learning_status { 0 } + is_suppressed { false } + is_available_at_library { true } + end + end + end +end diff --git a/spec/factories/leader.rb b/spec/factories/leader.rb new file mode 100644 index 0000000..d79b776 --- /dev/null +++ b/spec/factories/leader.rb @@ -0,0 +1,19 @@ +module Sierra + module Data + FactoryBot.define do + factory :leader, class: LeaderField do + id { 31 } + record_id { 420908636160 } + record_status_code { 'c' } + record_type_code { 'a' } + bib_level_code { 'm' } + control_type_code { ' ' } + char_encoding_scheme_code { ' ' } + encoding_level_code { 'I' } + descriptive_cat_form_code { 'a' } + multipart_level_code { ' ' } + base_address { 145 } + end + end + end +end diff --git a/spec/factories/location.rb b/spec/factories/location.rb new file mode 100644 index 0000000..a8007c5 --- /dev/null +++ b/spec/factories/location.rb @@ -0,0 +1,33 @@ +module Sierra + module Data + FactoryBot.define do + factory :loc, class: Location do + id { 277 } + branch_code_num { nil } + parent_location_code { nil } + is_public { false } + is_requestable { true } + + factory :loc_ddda do + code { 'ddda' } + end + + factory :loc_wbba do + code { 'wbba' } + end + + factory :loc_dd do + code { 'dd' } + end + + factory :loc_wb do + code { 'wb' } + end + + factory :loc_multi do + code { 'multi' } + end + end + end + end +end diff --git a/spec/factories/metadata.rb b/spec/factories/metadata.rb new file mode 100644 index 0000000..9966fbc --- /dev/null +++ b/spec/factories/metadata.rb @@ -0,0 +1,45 @@ +module Sierra + module Data + FactoryBot.define do + factory :metadata, class: Metadata do + id { 450974227090 } + record_num { 2661010 } + creation_date_gmt { Time.parse('2004-11-04 12:55:00 -0500') } + deletion_date_gmt { nil } + campus_code { '' } + agency_code_num { 0 } + num_revisions { 226 } + record_last_updated_gmt { Time.parse('2018-10-11 07:30:34 -0400') } + previous_last_updated_gmt { Time.parse('2017-06-30 21:41:00 -0400') } + + factory :metadata_deleted do + deletion_date_gmt { Time.parse('2004-11-04 12:55:00 -0500') } + end + + factory :metadata_a do + record_type_code { 'a' } + end + + factory :metadata_b do + record_type_code { 'b' } + end + + factory :metadata_c do + record_type_code { 'c' } + end + + factory :metadata_i do + record_type_code { 'i' } + end + + factory :metadata_o do + record_type_code { 'o' } + end + + factory :metadata_p do + record_type_code { 'p' } + end + end + end + end +end diff --git a/spec/factories/order.rb b/spec/factories/order.rb new file mode 100644 index 0000000..112f76f --- /dev/null +++ b/spec/factories/order.rb @@ -0,0 +1,51 @@ +module Sierra + module Data + FactoryBot.define do + factory :data_o, class: Order do + id { 476743101902 } + record_type_code { 'o' } + record_num { 1732046 } + creation_date_gmt { Time.parse('2015-07-02 09:53:00 -0400') } + deletion_date_gmt { nil } + campus_code { '' } + agency_code_num { '0' } + num_revisions { '8' } + record_last_updated_gmt { Time.parse('2015-07-10 13:54:08 -0400') } + previous_last_updated_gmt { Time.parse('2015-07-06 15:30:30 -0400') } + record_id { 476743101902 } + accounting_unit_code_num { 1 } + acq_type_code { 'p' } + catalog_date_gmt { Time.parse('2015-07-10 00:00:00 -0400') } + claim_action_code { 'n' } + ocode1 { '-' } + ocode2 { '-' } + ocode3 { '-' } + ocode4 { 'g' } + estimated_price { BigDecimal('0.0') } + form_code { 'a' } + order_date_gmt { Time.parse('2015-07-02 00:00:00 -0400') } + order_note_code { ' ' } + order_type_code { 'a' } + receiving_action_code { '-' } + received_date_gmt { Time.parse('2015-07-02 00:00:00 -0400') } + receiving_location_code { '3' } + billing_location_code { '3' } + order_status_code { 'a' } + temporary_location_code { 'c' } + vendor_record_code { 'ya11m' } + language_code { 'und' } + blanket_purchase_order_num { '' } + country_code { 'xx' } + volume_count { 1 } + fund_allocation_rule_code { nil } + reopen_text { '' } + list_price { nil } + list_price_foreign_amt { nil } + list_price_discount_amt { nil } + list_price_service_charge { nil } + is_suppressed { nil } + fund_copies_paid { nil } + end + end + end +end diff --git a/spec/factories/varfield.rb b/spec/factories/varfield.rb new file mode 100644 index 0000000..0f8da71 --- /dev/null +++ b/spec/factories/varfield.rb @@ -0,0 +1,79 @@ +module Sierra + module Data + FactoryBot.define do + factory :varfield, class: Varfield do + id { 114483 } + record_id { 450972566081 } + marc_ind1 { ' ' } + marc_ind2 { ' ' } + occ_num { 0 } + + factory :varfield_marc do + marc_tag { '245' } + marc_ind1 { '1' } + marc_ind2 { '0' } + varfield_type_code { 't' } + field_content { '|aSomething else :|ba novel' } + + factory :varfield_245 do + end + + factory :varfield_005 do + marc_tag { '005' } + varfield_type_code { 'y' } + field_content { '19820807000000.0' } + end + + factory :varfield_implicit_sfa do + field_content { 'Something else :|ba novel' } + end + end + + factory :varfield_i do + marc_tag { nil } + + factory :varfield_i_b do + varfield_type_code { 'b' } + field_content { '00050035567' } + end + + factory :varfield_i_c do + varfield_type_code { 'c' } + marc_tag { '090' } + field_content { '|aTR655|b.H66 2015' } + end + + factory :varfield_i_f do + varfield_type_code { 'f' } + field_content { 'ART' } + end + + factory :varfield_i_j do + varfield_type_code { 'j' } + field_content { 'VENDOR: YBP uncat' } + end + + factory :varfield_i_m do + varfield_type_code { 'm' } + field_content { 'Message' } + end + + factory :varfield_i_v do + varfield_type_code { 'v' } + field_content { 'Suppl.' } + end + + factory :varfield_i_x do + varfield_type_code { 'x' } + field_content { 'jc' } + end + + factory :varfield_i_z do + varfield_type_code { 'z' } + field_content { 'Second nature ; Reflections' } + end + end + end + end + end +end diff --git a/spec/helpers/varfields_spec.rb b/spec/helpers/varfields_spec.rb deleted file mode 100644 index b4efe99..0000000 --- a/spec/helpers/varfields_spec.rb +++ /dev/null @@ -1,135 +0,0 @@ -require_relative '../../lib/sierra_postgres_utilities.rb' - -class SpecDummy - include SierraPostgresUtilities::Helpers::Varfields -end - -RSpec.describe SierraPostgresUtilities::Helpers::Varfields do - let(:dummy) { SpecDummy.new } - - describe 'get_varfields' do - let(:bib) { SierraBib.new('b3260099a') } - let(:vf) { bib.get_varfields(['245b']) } - - it 'returns array' do - expect(vf).to be_an(Array) - end - - it 'returns array of hashed field representations' do - expect(vf[0]).to be_a(Hash) - end - - it 'adds extracted content value for each field' do - expect(vf[0]['extracted_content'][0]).to eq( - 'agriculture and education, planting the seeds of opportunity' - ) - end - end - - describe '#add_explicit_sf_a' do - let(:fc) { '|aIDEBK|beng|erda|cIDEBK|dCOO|aAAA' } - let(:fc_no_a) { 'IDEBK|beng|erda|cIDEBK|dCOO|aAAA' } - - context 'when field_content lacks initial "|a"' do - it 'adds explcit initial |a' do - expect(dummy.add_explicit_sf_a(fc_no_a)).to eq(fc) - end - - it 'does not modify original object' do - dummy.add_explicit_sf_a(fc_no_a) - expect(fc_no_a).to eq('IDEBK|beng|erda|cIDEBK|dCOO|aAAA') - end - end - - context 'when field_content lacks initial "|a"' do - it 'makes no changes' do - expect(dummy.add_explicit_sf_a(fc)).to eq(fc) - end - end - end - - describe '#subfield_from_field_content' do - let(:fc) { '|aIDEBK|beng|erda|cIDEBK|dCOO|aAAA' } - let(:fc_no_a) { 'IDEBK|beng|erda|cIDEBK|dCOO|aAAA' } - - it 'returns value of first matching subfield' do - expect(dummy.subfield_from_field_content(fc, 'a')). - to eq('IDEBK') - end - - context 'when implicit_sfa: true' do - it 'adds initial |a if content lacks initial sf delimiter' do - expect(dummy.subfield_from_field_content(fc_no_a, 'a', - implicit_sfa: true)). - to eq('IDEBK') - end - end - - it 'it implict_sfa is true by default' do - expect(dummy.subfield_from_field_content(fc_no_a, 'a')). - to eq(dummy.subfield_from_field_content(fc_no_a, 'a', - implicit_sfa: true)) - end - - context 'when implicit_sfa: false' do - it 'does not add initial |a when content lacks initial sf delimiter' do - expect(dummy.subfield_from_field_content(fc_no_a, 'a', - implicit_sfa: false)). - to eq('AAA') - end - end - end - - describe '#extract_subfields' do - - # accepts desired_subfields string - # accepts desired_subfield array of strings - # trim punct - # remove sf6880 - # implicit_sfa - - end - - describe '#subfield_arry' do - let(:fc) { '|aIDEBK|beng|erda|cIDEBK|dCOO|aAAA' } - let(:fc_arry) { - [["a", "IDEBK"], ["b", "eng"], ["e", "rda"], - ["c", "IDEBK"], ["d", "COO"], ["a", "AAA"]] - } - let(:fc_no_a) { 'IDEBK|beng|erda|cIDEBK|dCOO|aAAA' } - - it 'returns array of subfield,value pairs' do - expect(dummy.subfield_arry(fc)).to eq(fc_arry) - end - - context 'when implicit_sfa: true' do - it 'adds initial |a if content lacks initial sf delimiter' do - expect(dummy.subfield_arry(fc_no_a, implicit_sfa: true)). - to eq(fc_arry) - end - end - - it 'it implict_sfa is true by default' do - expect(dummy.subfield_arry(fc_no_a)). - to eq(dummy.subfield_arry(fc, implicit_sfa: true)) - end - - context 'when implicit_sfa: false' do - it 'does not add initial |a when content lacks initial sf delimiter' do - expect(dummy.subfield_arry(fc_no_a, implicit_sfa: false)). - to eq(fc_arry[1..-1]) - end - end - - context 'when subfield lacks subfield code' do - it 'discards that subfield' do - expect(dummy.subfield_arry('|adata||balso data')). - to eq([['a', 'data'], ['b', 'also data']]) - end - - it 'returns empty array if no subfields left' do - expect(dummy.subfield_arry('|')).to eq([]) - end - end - end -end diff --git a/spec/logging_spec.rb b/spec/logging_spec.rb new file mode 100644 index 0000000..8e70855 --- /dev/null +++ b/spec/logging_spec.rb @@ -0,0 +1,37 @@ +require 'spec_helper' + +module Sierra + RSpec.describe Logging do + let(:io) { StringIO.new } + let(:log) { Sierra.log_to(io) } + + describe '#log_to' do + it 'logs to passed object' do + log.warn('test') + expect(io.string).to match('test') + end + + it 'returns log' do + expect(log).to be_a(Logger) + end + end + + describe '#log_sql' do + it 'turns on logging of sql queries made' do + log + Sierra.log_sql + Sierra::Data::Bib.first + expect(io.string).to include('SELECT * FROM') + end + + context 'when passed false' do + it 'turns of logging of sql queries' do + log + Sierra.log_sql(false) + Sierra::Data::Bib.first + expect(io.string).not_to include('SELECT * FROM') + end + end + end + end +end diff --git a/spec/record_spec.rb b/spec/record_spec.rb new file mode 100644 index 0000000..53a3cd4 --- /dev/null +++ b/spec/record_spec.rb @@ -0,0 +1,57 @@ +require 'spec_helper' + +module Sierra + RSpec.describe Record do + context 'when record exists' do + describe '.get' do + let(:bnum) { 'b1191683a' } + let(:rec) { Sierra::Record.get(bnum) } + + it 'returns a Sierra::Data::[RecType] object for given bnum' do + expect(rec).to be_a(Sierra::Data::Bib) + end + + it 'bnum does not need a trailing "a"' do + bnum_short = 'b1191683' + expect(Sierra::Record.get(bnum_short)).to be_a(Sierra::Data::Bib) + end + + it 'can retrieve records by id' do + expect(Sierra::Record.get(id: 420907986691).rnum).to eq(bnum) + end + + it 'id can be given as a string' do + expect(Sierra::Record.get(id: '420907986691').rnum).to eq(bnum) + end + + # See Sierra::Data::Item code for details. If using natural joins + # in the model definition, retrieval of items with agency_code_num != 0 + # would fail. + it "retrieves items with non-zero agency_code_num's" do + i = Sierra::Record.get('i10195158a') + expect(i.agency_code_num != 0).to be true + end + end + + context 'but is deleted' do + let(:rec) { Sierra::Record.get('b6780003') } + + describe '#data' do + it 'returns a DeletedRecord' do + expect(rec).to be_a(Sierra::Data::DeletedRecord) + end + end + end + end + + context 'when record does not exist' do + let(:rec) { Sierra::Record.get('b00000000547475') } + + describe '.fetch' do + it 'raises an InvalidRecord error' do + expect { rec }.to raise_error(Sierra::Record::InvalidRecord) + end + end + end + end +end diff --git a/spec/records/sierra_authority_spec.rb b/spec/records/sierra_authority_spec.rb deleted file mode 100644 index eb061af..0000000 --- a/spec/records/sierra_authority_spec.rb +++ /dev/null @@ -1,37 +0,0 @@ -require_relative '../../lib/sierra_postgres_utilities.rb' - -def set_attr(obj, attr, value) - obj.instance_variable_set("@#{attr}", value) -end - -def mock_struct(hsh = {zzz: nil}) - Struct.new(*hsh.keys).new(*hsh.values) -end - - -RSpec.describe SierraAuthority do - let(:auth001) { SierraAuthority.new('a1500197a') } - - describe '#suppressed?' do - - context 'when #authority.record.is_suppressed' do - it 'returns true' do - set_attr(auth001, :authority_record, mock_struct( - id: 416613046253, - is_suppressed: true - )) - expect(auth001.suppressed?).to be true - end - end - - context 'when #authority.record.is_suppressed is false' do - it 'returns false' do - set_attr(auth001, :authority_record, mock_struct( - id: 416613046253, - is_suppressed: false - )) - expect(auth001.suppressed?).to be false - end - end - end -end diff --git a/spec/records/sierra_bib_spec.rb b/spec/records/sierra_bib_spec.rb deleted file mode 100644 index 9665005..0000000 --- a/spec/records/sierra_bib_spec.rb +++ /dev/null @@ -1,325 +0,0 @@ -require_relative '../../lib/sierra_postgres_utilities.rb' - -def set_attr(obj, attr, value) - obj.instance_variable_set("@#{attr}", value) -end - -def mock_struct(hsh = {zzz: nil}) - Struct.new(*hsh.keys).new(*hsh.values) -end - - -RSpec.describe SierraBib do - let(:bib001) { SierraBib.new('b1191683a') } - - describe 'initialize' do - sb1 = SierraBib.new('b1191683') - - it 'sets given bnum' do - expect(sb1.given_bnum).to eq('b1191683') - end - - it 'sets bnum' do - expect(sb1.bnum).to eq('b1191683a') - end - - it 'sets bib identifier' do - expect(sb1.record_id).to eq(420907986691) - end - - sb2 = SierraBib.new('b00000000547475') - it 'bib identifier is nil if bad bnum' do - expect(sb2.record_id).to eq(nil) - end - - it 'warn if bib identifier not retrieved' do - expect(sb2.warnings).to include( - 'No record was found in Sierra for this record number' - ) - end - - sb00 = SierraBib.new('b6780003') - it 'sets bib identifier if bib is deleted' do - expect(sb00.record_id).to eq(420913575011) - end - - it 'warn if bib deleted' do - expect(sb00.warnings).to include('This Sierra record was deleted') - end - - sb3 = SierraBib.new('bzq6780003') - it 'warn if bnum starts with letters other than b' do - expect(sb3.warnings).to include( - 'Cannot retrieve Sierra record. Rnum must start with b' - ) - end - - sb4 = SierraBib.new('b996780003') - it 'warn if bib identifier not retrieved' do - expect(sb4.warnings).to include( - 'No record was found in Sierra for this record number' - ) - end - -=begin -Note: -If we do: SierraBib.new('b9996780003') and try to set identifier, we will get a failure: - PG::NumericValueOutOfRange: - ERROR: value "9996780003" is out of range for type integer - LINE 4: and record_num = '9996780003' - -Shouldn't be a problem, so leaving it to fail in a nasty way for now. -=end - end - - describe '@bnum' do - it 'is set as bnum (including leading-b and trailing-a' do - expect(bib001.bnum).to eq('b1191683a') - end - end - - describe 'bnum_trunc' do - it 'yields bnum without check digit or "a"' do - expect(bib001.bnum_trunc).to eq('b1191683') - end - end - - describe 'bnum_with_check' do - it 'yields bnum including actual check digit' do - expect(bib001.bnum_with_check).to eq('b11916837') - end - end - - describe 'recnum' do - it 'yields recnum' do - expect(bib001.recnum).to eq('1191683') - end - end - - describe 'check_digit' do - it 'yields a string' do - expect(bib001.check_digit('1191683')).to be_an(String) - end - - it 'calculates check digit for a recnum' do - expect(bib001.check_digit('1191683')).to eq('7') - end - - it 'correctly calculates a check digit of "x"' do - expect(bib001.check_digit('1191693')).to eq('x') - end - end - - describe 'suppressed?' do - it 'returns true if bib is suppressed' do - bib = SierraBib.new('b5877843') - expect(bib.suppressed?).to eq(true) - end - - it 'returns false if bib is unsuppressed' do - bib = SierraBib.new('b3260099') - expect(bib.suppressed?).to eq(false) - end - - it 'counts bcode3 == "c" as suppressed' do - bib = SierraBib.new('b4576646') - expect(bib.suppressed?).to eq(true) - end - end - - context 'bib_record aliases' do - let(:bib) { SierraBib.new('b1841152a') } - - describe '#bcode1_blvl' do - end - - describe '#mat_type' do - end - end - - describe '#control_fields' do - let(:bib) { SierraBib.new('b3260099a') } - let(:cfs) { bib.control_fields } - - it 'includes control_fields stored in sierra_view.varfield' do - expect(cfs.select { |f| f[:marc_tag] == '001' }.empty?).to be false - end - - it 'includes 006/007/008 control_fields from sierra_view.control_field' do - expect(cfs.select { |f| f[:marc_tag] == '008' }.empty?).to be false - end - - it 'creates a field_content string from individual character positions' do - expect(cfs.select { |f| f[:marc_tag] == '008' }.first[:field_content]). - to eq('990707s1999 dcu f000 0 eng d') - end - - # For control fields, Sierra database stores a space for what should be a - # null position. (For example, p39 is meaningless/invalid for an 006, but - # the database will has p39 of ' ' rather than null.) - # 006s/008s are fixed length and we can assume any trailing spaces inside - # that length are part of the field - it 'does not rstrip 008s' do - set_attr( - bib, - :control_field, - [mock_struct( - :id=>1, :record_id=>420910055107, :varfield_type_code=>"y", - :control_num=>8, :p00=>"9", :p01=>"9", :p02=>"0", :p03=>"7", - :p04=>"0", :p05=>"7", :p06=>"s", :p07=>"1", :p08=>"9", :p09=>"9", - :p10=>"9", :p11=>" ", :p12=>" ", :p13=>" ", :p14=>" ", :p15=>"d", - :p16=>"c", :p17=>"u", :p18=>" ", :p19=>" ", :p20=>" ", :p21=>" ", - :p22=>" ", :p23=>" ", :p24=>" ", :p25=>" ", :p26=>" ", :p27=>" ", - :p28=>"f", :p29=>"0", :p30=>"0", :p31=>"0", :p32=>" ", :p33=>"0", - :p34=>" ", :p35=>"e", :p36=>"n", :p37=>" ", :p38=>" ", :p39=>" ", - :p40=>"c", :p41=>"a", :p42=>"m", :p43=>"7", :occ_num=>5, - :remainder=>"a " - )] - ) - expect(cfs.select { |f| f[:marc_tag] == '008' }.first[:field_content]). - to eq('990707s1999 dcu f000 0 en ') - end - - it 'does not rstrip 006s' do - expect(cfs.select { |f| f[:marc_tag] == '006' }.first[:field_content]). - to eq('m u f ') - end - - # 007s are variable length. We cannot trivially identify which trailing - # spaces are part of the actual 007 vs which are just part of the sierra db - # record, so we don't try to determine how many trailing spaces an 007 ought - # to have. - it 'rstrips 007s' do - set_attr( - bib, - :control_field, - [mock_struct( - :id=>1, :record_id=>420910055107, :varfield_type_code=>"y", - :control_num=>7, :p00=>"9", :p01=>"9", :p02=>"0", :p03=>"7", - :p04=>"0", :p05=>"7", :p06=>"s", :p07=>"1", :p08=>"9", :p09=>"9", - :p10=>"9", :p11=>" ", :p12=>" ", :p13=>" ", :p14=>" ", :p15=>"d", - :p16=>"c", :p17=>"u", :p18=>" ", :p19=>" ", :p20=>" ", :p21=>" ", - :p22=>" ", :p23=>" ", :p24=>" ", :p25=>" ", :p26=>" ", :p27=>" ", - :p28=>" ", :p29=>" ", :p30=>" ", :p31=>" ", :p32=>" ", :p33=>" ", - :p34=>" ", :p35=>" ", :p36=>" ", :p37=>" ", :p38=>" ", :p39=>" ", - :p40=>" ", :p41=>" ", :p42=>" ", :p43=>" ", :occ_num=>5, - :remainder=>"a " - )] - ) - expect(cfs.select { |f| f[:marc_tag] == '007' }.first[:field_content]). - to eq('990707s1999 dcu') - end - end - - describe '#ldr' do - let(:bib) { SierraBib.new('b1841152a') } - - it 'returns leader field as a string' do - expect(bib.ldr).to eq ('00000cam 2200145Ia 4500') - end - - it 'is 24 bytes/chars' do - expect(bib.ldr.length).to eq(24) - end - - it 'is nil when no leader field exists' do - set_attr(bib, :leader_field, OpenStruct.new) - expect(bib.ldr).to be nil - end - end - - context 'leader aliases' do - let(:bib) { SierraBib.new('b1841152a') } - - describe '#rec_type' do - it 'returns record type code' do - expect(bib.rec_type).to eq('a') - end - end - - describe '#blvl' do - it 'returns bib level code from leader' do - expect(bib.blvl).to eq('m') - end - end - - describe '#ctrl_type' do - it 'returns control type code' do - expect(bib.ctrl_type).to eq(' ') - end - end - end - - describe '#bib_locs' do - let(:bib) { SierraBib.new('b3439973') } - let(:locs) { bib.bib_locs } - - it 'returns array' do - expect(locs).to be_an(Array) - end - - it 'returns bib locations' do - expect(locs.include?('dd')).to be true - end - - it 'excludes "multi" as a bib location' do - expect(locs.length > 1 && !locs.include?('multi')).to be true - end - end - - describe '#best_title' do - let(:bib) { SierraBib.new('b1841152a') } - - it 'returns iii best_title' do - expect(bib.best_title).to eq('Something else : a novel') - end - end - - describe '#best_author' do - let(:bib) { SierraBib.new('b1841152a') } - - it 'returns iii best_author' do - expect(bib.best_author).to eq('Fassnidge, Virginia.') - end - end - - describe '#imprint' do - let(:bib) { SierraBib.new('b1841152a') } - - it 'returns cleaned value of first 260/264 field' do - expect(bib.imprint).to eq('London : Constable, 1981.') - end - end - - describe 'oclcnum' do - it 'gets oclcnum from MARC::Record' do - bib = SierraBib.new('b5244621') - expect(bib.oclcnum).to eq(bib.marc.oclcnum) - end - end - - context 'marc production' do - let(:bib) { SierraBib.new('b1841152a') } - let(:correct_mrc) { - m = MARC::Reader.new('spec/data/b1841152a.mrc').to_a.first - } - - describe '#marc' do - it 'returns a MARC::Record object' do - expect(bib.marc).to be_a(MARC::Record) - end - - it 'contains correct marc fields' do - expect(bib.marc.fields).to eq(correct_mrc.fields) - end - - it 'returns proper leader, apart from dummied fields/chars' do - bib.marc.leader[0..4] = '00000' - bib.marc.leader[12..16] = '00000' - correct_mrc.leader[0..4] = '00000' - correct_mrc.leader[12..16] = '00000' - expect(bib.marc.leader).to eq(correct_mrc.leader) - end - end - end -end diff --git a/spec/records/sierra_item_spec.rb b/spec/records/sierra_item_spec.rb deleted file mode 100644 index 07a56c8..0000000 --- a/spec/records/sierra_item_spec.rb +++ /dev/null @@ -1,155 +0,0 @@ -require_relative '../../lib/sierra_postgres_utilities.rb' - -class SierraItem - def set_varfield_data(hsh) - @varfield_data = hsh - end - - def set_checkout(hsh) - @checkout = Struct.new(*hsh.keys).new(*hsh.values) - end -end - -RSpec.describe SierraItem do - let(:item) { SierraItem.new('i2661010a') } - let(:item_no_vf) { SierraItem.new('i11136193a') } - let(:item_many_vf) { SierraItem.new('i10998994a') } - - describe '#inum_trunc' do - it 'returns without check digit or "a"' do - expect(item.inum_trunc).to eq('i2661010') - end - end - - describe '#barcodes' do - - it 'is empty when no "b" varfields' do - expect(item_no_vf.barcodes.empty?).to be true - end - - context 'by default (when value_only is explicitly true)' do - it 'returns barcodes as array of strings' do - expect(item.barcodes).to eq(['00001254305']) - end - end - - context 'when value_only is explicitly false' do - it 'returns barcodes as array of sql varfields' do - expect( - item.barcodes(value_only: false).first[:field_content] - ).to eq('00001254305') - end - end - end - - describe 'varfield retrieval by type' do - types = [ - {method: 'barcodes', values: ['00050035567']}, - {method: 'volumes', values: ['Suppl.']}, - {method: 'public_notes', values: ['Second nature ; Reflections']}, - {method: 'internal_notes', values: ["jc", "Shelf date 4/26/16 jhg"]}, - {method: 'stats_fields', values: ['VENDOR: YBP uncat']}, - {method: 'varfield_librarys', values: ['ART']}, - {method: 'callnos', values: ['TR655.H66 2015']} - ] - types.each do |type_hsh| - it "returns array of #{type_hsh[:method]} as strings" do - expect(item_many_vf.send(type_hsh[:method])).to eq(type_hsh[:values]) - end - end - end - - describe 'callnos' do - context 'when keep_delimiters: true' do - it 'leaves delimiters in the field contents' do - expect( - item_many_vf.callnos(keep_delimiters: true).first - ).to eq('|aTR655|b.H66 2015') - end - end - end - - describe '#icode2' do - it 'returns icode2' do - expect(item.icode2).to eq('-') - end - end - - describe '#itype_code' do - - # note that item_record.itype_code_num is a number - it 'returns itype code as a string' do - expect(item.itype_code).to eq('0') - end - end - - describe '#location_code' do - it 'returns location code' do - expect(item.location_code).to eq('trln') - end - end - - describe '#status_code' do - it 'returns status code' do - expect(item.status_code).to eq('-') - end - end - - describe '#copy_num' do - it 'returns copy num as a number' do - expect(item.copy_num).to eq(1) - end - end - - describe '#suppressed?' do - it 'returns boolean for suppression value' do - expect(item.suppressed?).to be false - end - end - - describe '#itype_description' do - it 'returns itype description / longname' do - expect(item.itype_description).to eq('Book') - end - end - - describe '#location_description' do - it 'returns location description / longname' do - expect( - item.location_description - ).to eq('Library Service Center — Request from Storage') - end - end - - describe '#status_description' do - it 'returns status description / longname' do - expect(item.status_description).to eq('Available') - end - end - - describe '#due_date' do - it 'returns due date as DateTime object' do - checked_item = SierraItem.new('i2661010a') - checked_item.set_checkout(due_gmt: Time.new(2018,8, 8, 4)) - expect(checked_item.due_date.is_a?(Time)).to be true - end - end - - describe '#is_oca?' do - - oca_book = SierraItem.new('i7364701a') - it 'returns true when book note present' do - expect(oca_book.is_oca?).to be true - end - - oca_journal = SierraItem.new('i7364813a') - it 'returns true when journal note present' do - expect(oca_journal.is_oca?).to be true - end - - non_oca = SierraItem.new('i1000035a') - it 'falsey if no oca note present' do - expect(non_oca.is_oca?).to be_falsey - end - end -end diff --git a/spec/records/sierra_record_spec.rb b/spec/records/sierra_record_spec.rb deleted file mode 100644 index 13469d6..0000000 --- a/spec/records/sierra_record_spec.rb +++ /dev/null @@ -1,76 +0,0 @@ -require_relative '../../lib/sierra_postgres_utilities.rb' - -def set_attr(obj, attr, value) - obj.instance_variable_set("@#{attr}", value) -end - - -RSpec.describe SierraRecord do - let(:rec) { SierraRecord.new(rnum: 'b1841152a', rtype: 'b') } - let(:del_rec) { SierraRecord.new(rnum: 'b6780003a', rtype: 'b') } - let(:invalid_rec) { SierraRecord.new(rnum: 'b1111111111a', rtype: 'b')} - - describe '#deleted?' do - - context 'record has been deleted' do - it 'returns boolean true' do - expect(del_rec.deleted?).to be true - end - end - - context 'record has not been deleted' do - it 'returns falsey' do - expect(rec.deleted?).to be_falsey - end - end - - context 'record is invalid' do - it 'returns falsey' do - expect(invalid_rec.deleted?).to be_falsey - end - end - end - - describe '#invalid?' do - - context 'record never existed' do - it 'returns boolean true' do - expect(invalid_rec.invalid?).to be true - end - end - - context 'undeleted record exists' do - it 'returns falsey' do - expect(rec.invalid?).to be_falsey - end - end - - context 'deleted record exists' do - it 'returns falsey' do - expect(del_rec.invalid?).to be_falsey - end - end - - - end - - describe '.vf_codes' do - it 'returns hash of type_codes:type_names' do - expect(SierraItem.vf_codes['b']).to eq('Barcode') - end - - it 'uses varfield_type_name.short_name when name is empty' do - expect(SierraItem.vf_codes['8']).to eq('HOLD') - end - end - - describe '#vf_codes' do - it 'returns hash of type_codes:type_names' do - expect(rec.vf_codes['c']).to eq('Call No.') - end - - it 'uses varfield_type_name.short_name when name is empty' do - expect(rec.vf_codes['8']).to eq('HOLD') - end - end -end diff --git a/spec/search_spec.rb b/spec/search_spec.rb new file mode 100644 index 0000000..90a297d --- /dev/null +++ b/spec/search_spec.rb @@ -0,0 +1,55 @@ +require 'spec_helper' + +module Sierra::Search + RSpec.describe PhraseSearch do + it 'finds record by ocn' do + expect(PhraseSearch.phrase_search(:o, 'ccn00828088')&. + first&. + record_id). + to eq(420915771633) + end + + it 'ocn search is case-insenstive' do + expect(PhraseSearch.phrase_search(:o, 'CCN00828088')&. + first&. + record_id). + to eq(420915771633) + end + + ocns = [ + ['5916808|%1762189', 420908463899], + ['B-1565-11', 420911493821], + ['nchg2df651fe-7079-47a2-b29d-77ea90702dc1', 420915817690], + ['|z60618545', 420912632268], + ['32988245|z(OCoLC)62441777|z(OCoLC)77633390|z(OCoLC)77633393|z(OCoLC)77664829|z(OCoLC)77664832|z(OCoLC)77734146|z(OCoLC)77768001|z(OCoLC)77819922|z(OCoLC)77867473|z(OCoLC)77879873|z(OCoLC)77948921|z(OCoLC)77992118|z(OCoLC)78156448|z(OCoLC)78202144|z(OCoLC)78227074|z(OCoLC)78229478|z(OCoLC)78263835|z(OCoLC)78294354|z(OCoLC)78356784|z(OCoLC)78356786|z(OCoLC)78378278|z(OCoLC)78494104|z(OCoLC)78494106|z(OCoLC)78546807|z(OCoLC)78644222|z(OCoLC)78696909|z(OCoLC)78731572|z(OCoLC)78773090|z(OCoLC)78773094|z(OCoLC)78823416|z(OCoLC)78849123|z(OCoLC)78849128|z(OCoLC)78850176|z(OCoLC)78850178|z(OCoLC)78954175|z(OCoLC)78954180|z(OCoLC)79018046|z(OCoLC)79019238|z(OCoLC)79023976|z(OCoLC)79079593|z(OCoLC)79129035|z(OCoLC)79327189|z(OCoLC)79619320|z(OCoLC)79619321|z(OCoLC)79657868|z(OCoLC)79667407|z(OCoLC)79717203|z(OCoLC)79760843|z(OCoLC)79792915|z(OCoLC)79919861|z(OCoLC)79919870|z(OCoLC)79982952|z(OCoLC)79982954|z(OCoLC)80046751|z(OCoLC)80098487|z(OCoLC)80098490|z(OCoLC)80172878|z(OCoLC)80303652|z(OCoLC)80340445|z(OCoLC)80402874|z(OCoLC)80624299|z(OCoLC)80635972|z(OCoLC)80712023|z(OCoLC)80712027|z(OCoLC)80719584|z(OCoLC)80781063|z(OCoLC)81003423|z(OCoLC)81003427|z(OCoLC)81064327|z(OCoLC)81064351|z(OCoLC)81064353|z(OCoLC)81071054|z(OCoLC)81275351|z(OCoLC)81275354|z(OCoLC)81333101|z(OCoLC)81363913|z(OCoLC)81410442|z(OCoLC)81475533|z(OCoLC)81476337|z(OCoLC)81476340|z(OCoLC)81511140|z(OCoLC)81642668|z(OCoLC)81678817|z(OCoLC)81678826|z(OCoLC)81719138|z(OCoLC)81794381|z(OCoLC)81794382|z(OCoLC)81821005|z(OCoLC)81868540|z(OCoLC)81952264|z(OCoLC)81992783|z(OCoLC)82015486|z(OCoLC)82031277|z(OCoLC)82114916|z(OCoLC)82167627|z(OCoLC)82212378|z(OCoLC)82212514|z(OCoLC)82307779|z(OCoLC)82333563', 420913252173] + ] + ocns.each do |ocn, record_id| + it "matches ocn: #{ocn} to rec: #{record_id}" do + expect(PhraseSearch.phrase_search(:o, ocn)&. + first&. + record_id). + to eq(record_id) + end + end + + it 'returns nil when normalized search term is empty' do + expect(PhraseSearch.phrase_search(:o, ' ')&.first&.record_id). + to be_nil + end + + xit 'search terms including Han characters succeed' do + # Han characters stored as kCCCII values in phrase_entry and + # our search ought to properly translate them + end + + xit 'searching a 10-digit ISBN returns recs where Sierra has only a 13' do + # and vice versa + end + + xit 'search rec_type specification works' do + end + + xit 'search match strategy specification works' do + end + end +end diff --git a/spec/spec_data/b1841152a.altmarc.xml b/spec/spec_data/b1841152a.altmarc.xml new file mode 100644 index 0000000..cdba303 --- /dev/null +++ b/spec/spec_data/b1841152a.altmarc.xml @@ -0,0 +1,34 @@ + + 00469cam 2200169Ia 4500 + b1841152 + NcU + 19820807000000.0 + 820807s1981 enk 000 1 eng d + + 0094643407 + + + (OCoLC)8671134 + + + NOC + NOC + + + Fassnidge, Virginia. + + + Something else : + a novel / + Virginia Fassnidge. + + + London : + Constable, + 1981. + + + 152 p. ; + 23 cm. + + diff --git a/spec/data/b1841152a.mrc b/spec/spec_data/b1841152a.mrc similarity index 100% rename from spec/data/b1841152a.mrc rename to spec/spec_data/b1841152a.mrc diff --git a/spec/spec_data/b1841152a.xml b/spec/spec_data/b1841152a.xml new file mode 100644 index 0000000..e9ff11f --- /dev/null +++ b/spec/spec_data/b1841152a.xml @@ -0,0 +1,40 @@ + + 00469cam 2200169Ia 4500 + 8671134 + 19820807000000.0 + 820807s1981 enk 000 1 eng d + + 0094643407 + + + NOC + NOC + + + Fassnidge, Virginia. + + + Something else : + a novel / + Virginia Fassnidge. + + + London : + Constable, + 1981. + + + 152 p. ; + 23 cm. + + + .b18411526 + + + Baseline 09_2013 + Under Authority Control + + + ADH-2114 + + diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb new file mode 100644 index 0000000..e70189b --- /dev/null +++ b/spec/spec_helper.rb @@ -0,0 +1,38 @@ +$LOAD_PATH.unshift File.expand_path('../lib', __dir__) +require 'rspec' +require 'sierra_postgres_utilities' +require 'factory_bot' + +RSpec.configure do |config| + config.include FactoryBot::Syntax::Methods + FactoryBot.find_definitions +end + +def dummy_set(hsh) + hsh.each do |k, v| + dummy.data.send(:"#{k}=", v) + end + dummy +end + +module Sierra + module SpecUtils + module Records + def values=(hsh) + @values = hsh + end + + def set_data(field, data) + define_singleton_method(field) { data } + self + end + end + end +end + +def newrec(type, metadata = {}, data = {}) + rec = type.new + rec.extend(Sierra::SpecUtils::Records) + rec.values = metadata.to_hash.merge(data.to_hash) + rec +end