relaton/relaton-nist

= Relaton for NIST: bibliographic retrieval of NIST publications

image:https://img.shields.io/gem/v/relaton-nist.svg["Gem Version", link="https://rubygems.org/gems/relaton-nist"]
image:https://github.com/relaton/relaton-nist/workflows/macos/badge.svg["Build Status (macOS)", link="https://github.com/relaton/relaton-nist/actions?workflow=macos"]
image:https://github.com/relaton/relaton-nist/workflows/windows/badge.svg["Build Status (Windows)", link="https://github.com/relaton/relaton-nist/actions?workflow=windows"]
image:https://github.com/relaton/relaton-nist/workflows/ubuntu/badge.svg["Build Status (Ubuntu)", link="https://github.com/relaton/relaton-nist/actions?workflow=ubuntu"]
image:https://codeclimate.com/github/relaton/relaton-nist/badges/gpa.svg["Code Climate", link="https://codeclimate.com/github/relaton/relaton-nist"]
image:https://img.shields.io/github/issues-pr-raw/relaton/relaton-nist.svg["Pull Requests", link="https://github.com/relaton/relaton-nist/pulls"]
image:https://img.shields.io/github/commits-since/relaton/relaton-nist/latest.svg["Commits since latest",link="https://github.com/relaton/relaton-nist/releases"]

== Purpose

`relaton-nist` provides bibliographic information of NIST publications using the
https://github.com/metanorma/metanorma-model-nist#nist-bibliographic-item-model[NistBibliographicItem model].

Relaton for NIST has been developed in cooperation with the NIST Cybersecurity
Resource Center (CSRC) and the Computer Security Division (ITL/CSD).

== Data sources

Relaton for NIST retrieves bibliographic information from two sources:

* bibliographic feed from the NIST Cybersecurity Resource Center (CSRC) of
all CSRC publications (in Relaton JSON)
* bibliographic dataset from the NIST Library through the Information Services
Office (ISO) that contains information about all NIST Technical Publications
(https://github.com/usnistgov/NIST-Tech-Pubs[GitHub])

Bibliographic information offered through CSRC is provided with enhanced
metadata that is not available in the NIST Library dataset, including:

* public drafts (the NIST Library dataset only contains final publications)
* revision information: revision number, iteration
* document stage information: retired, withdrawn, etc.
* bibliographic dates, including issued date, updated date, published date,
  obsolete date, commenting period
* document relationships: supersession and replacements
* contacts: enhanced name parts and affiliation information

Relaton for NIST, therefore, uses the following order of priority for the data
sources:

. bibliographic feed from NIST CSRC
. NIST Library dataset

The NIST CSRC provides:

* NIST SP 800-*
* NIST SP 500-*
* NIST SP 1800-*
* NIST FIPS {31, 39, 41, 46, 46-$$*$$, 48, 65, 73, 74, 81, 83, 87, 102, 112, 113, 139, 140-$$*$$, 141, 171, 180, 180-$$*$$, 181, 186, 186-$$*$$, 188, 190, 191, 196, 197, 198, 198-1, 199, 200, 201, 201-*, 202}

The NIST Library dataset provides documents listed in the https://github.com/relaton/relaton-data-nist/blob/main/index-v1.yaml[index].

== Installation

Add this line to your application's Gemfile:

[source,ruby]
----
gem 'relaton-nist'
----

And then execute:

    $ bundle

Or install it yourself as:

    $ gem install relaton_nist

== Usage

=== Configuration

Configuration is optional. The available option is `logger` which is a `Logger` instance. By default, the logger is `Logger.new($stderr)` with `Logger::WARN` level. To change the logger level, use `RelatonNist.configure` block.

[source,ruby]
----
require 'relaton_nist'
=> true

RelatonNist.configure do |config|
  config.logger.level = Logger::DEBUG
end
----

=== Search for a standard using keywords

[source,ruby]
----
hit_collection = RelatonNist::NistBibliography.search("NISTIR 8200")
[relaton-nist] (NIST IR 8200) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST IR 8200) Fetching from Relaton repository ...
=> <RelatonNist::HitCollection:0x00000000004b28 @ref=NIST IR 8200 @fetched=false>

item = hit_collection[0].fetch
=> #<RelatonNist::NistBibliographicItem:0x007fc049aa6778
 ...
----

=== XML serialization
[source,ruby]
----
item.to_xml
=> "<bibitem id="NISTIR8200" type="standard" schema-version="v1.2.8">
      <fetched>2023-10-16</fetched>
      <title format="text/plain" language="en" script="Latn">Interagency report on the status of international cybersecurity standardization for the internet of things (IoT)</title>
      ...
    <bibitem>"
----
With argument `bibdata: true` it outputs XML wrapped by `bibdata` element and adds flavor `ext` element.
[source,ruby]
----
item.to_xml bibdata: true
=> "<bibdata type="standard" schema-version="v1.2.8">
      <fetched>2023-10-16</fetched>
      <title format="text/plain" language="en" script="Latn">Interagency report on the status of international cybersecurity standardization for the internet of things (IoT)</title>
      ...
      <ext schema-version="v1.0.0">
        <doctype>standard</doctype>
      </ext>
  </bibdata>"
----

=== Get code, and year
[source,ruby]
----
RelatonNist::NistBibliography.get("NIST IR 8200", "2018")
[relaton-nist] (NIST SP 8200:2018) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST IR 8200:2018) Fetching from Relaton repository...
[relaton-nist] (NIST IR 8200:2018) Found: `NIST IR 8200`
=> #<RelatonNist::NistBibliographicItem:0x00007fab74a572c0
...
----

=== Get short citation
A short citation is a convention about a citation's format. The format for NIST publications is:
----
NIST {abbrev(series)} {docnumber} {(edition), optional} {(stage), optional}
# or
{abbrev(series)} {docnumber} {(edition), optional} {(stage), optional}
----
- `(stage)` is empty if the state is "final" (published). In case the state is "draft" it should be:
  * PD for public draft
  * IPD for initial iteration public draft or 2PD, 3PD and so one for following iterations
  * FPD for final public draft
- `(edition)` is the date of publication or update
- `docnumber` is the full NIST number, including revision, e.g., 800-52

The format for FIPS publications is:
----
FIPS {docnumber}
# or
NIST FIPS {docnumber}
----
[source,ruby]
----
RelatonNist::NistBibliography.get("NIST SP 800-205 (February 2019) (IPD)")
[relaton-nist] (NIST SP 800-205) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST SP 800-205) Found: `NIST SP 800-205 (Draft)`
=> #<RelatonNist::NistBibliographicItem:0x00000001105afdc8
...
----

=== Get specific part, volume, version, revision, and addendum

Referehces can contain optional parameters `{ptN}{vN}{verN}{rN}{/Add}`:
- Part is specified as `ptN` (SP 800-57pt1)
- Volume is specified as `vN` (SP 800-60v1)
- Version is specified as `verN` (SP 800-45ver2)
- Revision is specified as `rN` (SP 800-40r3)
- Addendum is specified as `/Add` (SP 800-38A/Add)

[source,ruby]
----
item = RelatonNist::NistBibliography.get 'NIST SP 800-67r1'
[relaton-nist] (NIST SP 800-67r1) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST SP 800-67r1) Found: `NIST SP 800-67 Rev. 1`
=> #<RelatonNist::NistBibliographicItem:0x00000001105acd08
...

item.docidentifier.first.id
=> "NIST SP 800-67 Rev. 1"

item = RelatonNist::NistBibliography.get 'NIST SP 800-38A/Add'
[relaton-nist] (NIST SP 800-38A/Add) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST SP 800-38A/Add) Found: `NIST SP 800-38A-Add`
=> #<RelatonNist::NistBibliographicItem:0x00000001105abf48
...

item.docidentifier.first.id
=> "NIST SP 800-38A-Add"
----

=== Typed links

NIST documents may have `src` and `doi` link types.

[source,ruby]
----
item.link
=> [#<RelatonBib::TypedUri:0x0000000111087a98
  @content=#<Addressable::URI:0x8c0 URI:https://csrc.nist.gov/pubs/sp/800/38/a/sup/final>,
  @language=nil,
  @script=nil,
  @type="src">,
 #<RelatonBib::TypedUri:0x00000001110879a8 @content=#<Addressable::URI:0x8d4 URI:https://doi.org/10.6028/NIST.SP.800-38A-Add>, @language=nil, @script=nil, @type="doi">]
----

=== Create bibliographic item from YAML
[source,ruby]
----
hash = YAML.load_file 'spec/examples/nist_bib_item.yml'
=> {"id"=>"NISTIR 8011 Vol. 3",
...

RelatonNist::NistBibliographicItem.from_hash hash
=> #<RelatonNist::NistBibliographicItem:0x007f8b708505b8
...
----

=== Fetch data

This gem uses the https://raw.githubusercontent.com/usnistgov/NIST-Tech-Pubs/nist-pages/xml/allrecords.xml dataset as one of data sources.

The method `RelatonNist::DataFetcher.fetch(output: "data", format: "yaml")` fetches all the documents from the datast and save them to the `./data` folder in YAML format.
Arguments:

- `output` - folder to save documents (default './data').
- `format` - the format in which the documents are saved. Possible formats are: `yaml`, `xml`, `bibxxml` (default `yaml`).

[source,ruby]
----
RelatonNist::DataFetcher.fetch
Started at: 2021-09-01 18:01:01 +0200
Stopped at: 2021-09-01 18:01:43 +0200
Done in: 42 sec.
=> nil
----

== Development

After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).

== Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/metanorma/relaton-nist.

== License

The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).