Skip to content

Memory optimisation & rubocop on tests #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 19, 2023
Merged

Memory optimisation & rubocop on tests #59

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
09f9917
Enhance performance
abrisse Apr 17, 2023
fa0846b
chore: Enhance lisibility of some files
abrisse Apr 17, 2023
072c89d
chore: Enhance tests lisibility
abrisse Apr 17, 2023
c9e4542
Optimize Context memory usage
abrisse Apr 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .rubocop.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
require:
- rubocop-performance
- rubocop-rspec

AllCops:
DisplayCopNames: true
Expand Down
71 changes: 37 additions & 34 deletions Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,50 +1,53 @@
# frozen_string_literal: true

source "https://rubygems.org"
gem "nokogiri", '~> 1.13', '>= 1.13.4'
gem "nokogiri", '~> 1.13', '>= 1.13.4'

gemspec
gem 'rdf', git: "https://github.com/ruby-rdf/rdf", branch: "develop"
gem 'json-canonicalization',git: "https://github.com/dryruby/json-canonicalization",branch: "develop"
gem 'json-canonicalization', git: "https://github.com/dryruby/json-canonicalization", branch: "develop"
gem 'rdf', git: "https://github.com/ruby-rdf/rdf", branch: "develop"

group :development do
gem 'ebnf', git: "https://github.com/dryruby/ebnf", branch: "develop"
gem 'json-ld-preloaded', github: "ruby-rdf/json-ld-preloaded", branch: "develop"
gem 'ld-patch', github: "ruby-rdf/ld-patch", branch: "develop"
gem 'linkeddata', git: "https://github.com/ruby-rdf/linkeddata", branch: "develop"
gem 'rack-linkeddata', git: "https://github.com/ruby-rdf/rack-linkeddata", branch: "develop"
gem 'rdf-aggregate-repo', git: "https://github.com/ruby-rdf/rdf-aggregate-repo", branch: "develop"
gem 'rdf-isomorphic', git: "https://github.com/ruby-rdf/rdf-isomorphic", branch: "develop"
gem 'rdf-json', github: "ruby-rdf/rdf-json", branch: "develop"
gem 'rdf-microdata', git: "https://github.com/ruby-rdf/rdf-microdata", branch: "develop"
gem 'rdf-n3', github: "ruby-rdf/rdf-n3", branch: "develop"
gem 'rdf-normalize', github: "ruby-rdf/rdf-normalize", branch: "develop"
gem 'rdf-rdfa', git: "https://github.com/ruby-rdf/rdf-rdfa", branch: "develop"
gem 'rdf-rdfxml', git: "https://github.com/ruby-rdf/rdf-rdfxml", branch: "develop"
gem 'rdf-reasoner', github: "ruby-rdf/rdf-reasoner", branch: "develop"
gem 'rdf-spec', git: "https://github.com/ruby-rdf/rdf-spec", branch: "develop"
gem 'rdf-tabular', github: "ruby-rdf/rdf-tabular", branch: "develop"
gem 'rdf-trig', git: "https://github.com/ruby-rdf/rdf-trig", branch: "develop"
gem 'rdf-trix', github: "ruby-rdf/rdf-trix", branch: "develop"
gem 'rdf-turtle', git: "https://github.com/ruby-rdf/rdf-turtle", branch: "develop"
gem 'rdf-vocab', git: "https://github.com/ruby-rdf/rdf-vocab", branch: "develop"
gem 'rdf-xsd', git: "https://github.com/ruby-rdf/rdf-xsd", branch: "develop"
gem 'sinatra-linkeddata', git: "https://github.com/ruby-rdf/sinatra-linkeddata", branch: "develop"
gem 'shex', github: "ruby-rdf/shex", branch: "develop"
gem 'sparql', git: "https://github.com/ruby-rdf/sparql", branch: "develop"
gem 'sparql-client', git: "https://github.com/ruby-rdf/sparql-client", branch: "develop"
gem 'sxp', git: "https://github.com/dryruby/sxp.rb", branch: "develop"
gem 'fasterer'
gem 'earl-report'
gem 'ruby-prof', platforms: :mri
gem 'ebnf', git: "https://github.com/dryruby/ebnf", branch: "develop"
gem 'fasterer'
gem 'json-ld-preloaded', github: "ruby-rdf/json-ld-preloaded", branch: "develop"
gem 'ld-patch', github: "ruby-rdf/ld-patch", branch: "develop"
gem 'linkeddata', git: "https://github.com/ruby-rdf/linkeddata", branch: "develop"
gem 'rack-linkeddata', git: "https://github.com/ruby-rdf/rack-linkeddata", branch: "develop"
gem 'rdf-aggregate-repo', git: "https://github.com/ruby-rdf/rdf-aggregate-repo", branch: "develop"
gem 'rdf-isomorphic', git: "https://github.com/ruby-rdf/rdf-isomorphic", branch: "develop"
gem 'rdf-json', github: "ruby-rdf/rdf-json", branch: "develop"
gem 'rdf-microdata', git: "https://github.com/ruby-rdf/rdf-microdata", branch: "develop"
gem 'rdf-n3', github: "ruby-rdf/rdf-n3", branch: "develop"
gem 'rdf-normalize', github: "ruby-rdf/rdf-normalize", branch: "develop"
gem 'rdf-rdfa', git: "https://github.com/ruby-rdf/rdf-rdfa", branch: "develop"
gem 'rdf-rdfxml', git: "https://github.com/ruby-rdf/rdf-rdfxml", branch: "develop"
gem 'rdf-reasoner', github: "ruby-rdf/rdf-reasoner", branch: "develop"
gem 'rdf-spec', git: "https://github.com/ruby-rdf/rdf-spec", branch: "develop"
gem 'rdf-tabular', github: "ruby-rdf/rdf-tabular", branch: "develop"
gem 'rdf-trig', git: "https://github.com/ruby-rdf/rdf-trig", branch: "develop"
gem 'rdf-trix', github: "ruby-rdf/rdf-trix", branch: "develop"
gem 'rdf-turtle', git: "https://github.com/ruby-rdf/rdf-turtle", branch: "develop"
gem 'rdf-vocab', git: "https://github.com/ruby-rdf/rdf-vocab", branch: "develop"
gem 'rdf-xsd', git: "https://github.com/ruby-rdf/rdf-xsd", branch: "develop"
gem 'ruby-prof', platforms: :mri
gem 'shex', github: "ruby-rdf/shex", branch: "develop"
gem 'sinatra-linkeddata', git: "https://github.com/ruby-rdf/sinatra-linkeddata", branch: "develop"
gem 'sparql', git: "https://github.com/ruby-rdf/sparql", branch: "develop"
gem 'sparql-client', git: "https://github.com/ruby-rdf/sparql-client", branch: "develop"
gem 'sxp', git: "https://github.com/dryruby/sxp.rb", branch: "develop"
end

group :development, :test do
gem 'simplecov', '~> 0.21', platforms: :mri
gem 'simplecov-lcov', '~> 0.8', platforms: :mri
gem 'psych', platforms: [:mri, :rbx]
gem 'benchmark-ips'
gem 'psych', platforms: %i[mri rbx]
gem 'rake'
gem 'rubocop'
gem 'rubocop-performance'
gem 'rubocop-rspec'
gem 'simplecov', '~> 0.21', platforms: :mri
gem 'simplecov-lcov', '~> 0.8', platforms: :mri
end

group :debug do
Expand Down
71 changes: 39 additions & 32 deletions Rakefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# frozen_string_literal: true

require 'rubygems'

task default: [ :spec ]
task default: [:spec]

namespace :gem do
desc "Build the json-ld-#{File.read('VERSION').chomp}.gem file"
Expand All @@ -17,18 +19,18 @@ end
require 'rspec/core/rake_task'
desc 'Run specifications'
RSpec::Core::RakeTask.new(:spec) do |spec|
spec.rspec_opts = %w(--options spec/spec.opts) if File.exists?('spec/spec.opts')
spec.rspec_opts = %w[--options spec/spec.opts] if File.exist?('spec/spec.opts')
end

desc "Generate schema.org context"
task :schema_context do
%x(
`
script/gen_context https://schema.org/docs/schema_org_rdfa.html \
--vocab http://schema.org/ \
--prefix 'schema http://schema.org/' \
--body --hier \
--o etc/schema.org.jsonld
)
`
end

desc "Create concatenated test manifests"
Expand All @@ -37,68 +39,73 @@ file "etc/manifests.nt" do
require 'json/ld'
require 'rdf/ntriples'
graph = RDF::Graph.new do |g|
%w( https://w3c.github.io/json-ld-api/tests/compact-manifest.jsonld
%w[ https://w3c.github.io/json-ld-api/tests/compact-manifest.jsonld
https://w3c.github.io/json-ld-api/tests/expand-manifest.jsonld
https://w3c.github.io/json-ld-api/tests/flatten-manifest.jsonld
https://w3c.github.io/json-ld-api/tests/fromRdf-manifest.jsonld
https://w3c.github.io/json-ld-api/tests/html-manifest.jsonld
https://w3c.github.io/json-ld-api/tests/remote-doc-manifest.jsonld
https://w3c.github.io/json-ld-api/tests/toRdf-manifest.jsonld
https://w3c.github.io/json-ld-framing/tests/frame-manifest.jsonld
).each do |man|
https://w3c.github.io/json-ld-framing/tests/frame-manifest.jsonld].each do |man|
puts "load #{man}"
g.load(man, unique_bnodes: true)
end
end
puts "write"
RDF::NTriples::Writer.open("etc/manifests.nt", unique_bnodes: true, validate: false) {|w| w << graph}
RDF::NTriples::Writer.open("etc/manifests.nt", unique_bnodes: true, validate: false) { |w| w << graph }
end

# Presentation building
namespace :presentation do
desc "Clean presentation files"
task :clean do
FileUtils.rm %w(compacted expanded framed).map {|f| "presentation/dbpedia/#{f}.jsonld"}
FileUtils.rm %w[compacted expanded framed].map { |f| "presentation/dbpedia/#{f}.jsonld" }
end

desc "Build presentation files"
task build: %w(
task build: %w[
presentation/dbpedia/expanded.jsonld
presentation/dbpedia/compacted.jsonld
presentation/dbpedia/framed.jsonld
)
]

desc "Build expanded example"
file "presentation/dbpedia/expanded.jsonld" => %w(
file "presentation/dbpedia/expanded.jsonld" => %w[
presentation/dbpedia/orig.jsonld
presentation/dbpedia/expanded-context.jsonld) do
system(%w(
script/parse
--expand presentation/dbpedia/orig.jsonld
--context presentation/dbpedia/expanded-context.jsonld
-o presentation/dbpedia/expanded.jsonld).join(" "))
presentation/dbpedia/expanded-context.jsonld
] do
system(%w[
script/parse
--expand presentation/dbpedia/orig.jsonld
--context presentation/dbpedia/expanded-context.jsonld
-o presentation/dbpedia/expanded.jsonld
].join(" "))
end

desc "Build compacted example"
file "presentation/dbpedia/compacted.jsonld" => %w(
file "presentation/dbpedia/compacted.jsonld" => %w[
presentation/dbpedia/expanded.jsonld
presentation/dbpedia/compact-context.jsonld) do
system(%w(
script/parse
--compact presentation/dbpedia/expanded.jsonld
--context presentation/dbpedia/compact-context.jsonld
-o presentation/dbpedia/compacted.jsonld).join(" "))
presentation/dbpedia/compact-context.jsonld
] do
system(%w[
script/parse
--compact presentation/dbpedia/expanded.jsonld
--context presentation/dbpedia/compact-context.jsonld
-o presentation/dbpedia/compacted.jsonld
].join(" "))
end

desc "Build framed example"
file "presentation/dbpedia/framed.jsonld" => %w(
file "presentation/dbpedia/framed.jsonld" => %w[
presentation/dbpedia/expanded.jsonld
presentation/dbpedia/frame.jsonld) do
system(%w(
script/parse
--frame presentation/dbpedia/frame.jsonld
presentation/dbpedia/expanded.jsonld
-o presentation/dbpedia/framed.jsonld).join(" "))
presentation/dbpedia/frame.jsonld
] do
system(%w[
script/parse
--frame presentation/dbpedia/frame.jsonld
presentation/dbpedia/expanded.jsonld
-o presentation/dbpedia/framed.jsonld
].join(" "))
end
end

Expand Down
41 changes: 20 additions & 21 deletions json-ld.gemspec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,47 +1,44 @@
#!/usr/bin/env ruby -rubygems
# -*- encoding: utf-8 -*-
# frozen_string_literal: true

is_java = RUBY_PLATFORM == 'java'

Gem::Specification.new do |gem|
gem.version = File.read('VERSION').chomp
gem.date = File.mtime('VERSION').strftime('%Y-%m-%d')

gem.name = "json-ld"
gem.homepage = "https://github.com/ruby-rdf/json-ld"
gem.license = 'Unlicense'
gem.summary = "JSON-LD reader/writer for Ruby."
gem.description = "JSON::LD parses and serializes JSON-LD into RDF and implements expansion, compaction and framing API interfaces for the Ruby RDF.rb library suite."
gem.metadata = {
gem.metadata = {
"documentation_uri" => "https://ruby-rdf.github.io/json-ld",
"bug_tracker_uri" => "https://github.com/ruby-rdf/json-ld/issues",
"homepage_uri" => "https://github.com/ruby-rdf/json-ld",
"mailing_list_uri" => "https://lists.w3.org/Archives/Public/public-rdf-ruby/",
"source_code_uri" => "https://github.com/ruby-rdf/json-ld",
"bug_tracker_uri" => "https://github.com/ruby-rdf/json-ld/issues",
"homepage_uri" => "https://github.com/ruby-rdf/json-ld",
"mailing_list_uri" => "https://lists.w3.org/Archives/Public/public-rdf-ruby/",
"source_code_uri" => "https://github.com/ruby-rdf/json-ld",
'rubygems_mfa_required' => 'true'
}

gem.authors = ['Gregg Kellogg']
gem.email = 'public-linked-json@w3.org'

gem.platform = Gem::Platform::RUBY
gem.files = %w(AUTHORS README.md UNLICENSE VERSION) + Dir.glob('lib/**/*.rb')
gem.bindir = %q(bin)
gem.executables = %w(jsonld)
gem.require_paths = %w(lib)
gem.test_files = Dir.glob('spec/**/*.rb') + Dir.glob('spec/test-files/*')
gem.files = %w[AUTHORS README.md UNLICENSE VERSION] + Dir.glob('lib/**/*.rb')
gem.bindir = 'bin'
gem.executables = %w[jsonld]
gem.require_paths = %w[lib]

gem.required_ruby_version = '>= 2.6'
gem.requirements = []
gem.add_runtime_dependency 'rdf', '~> 3.2', '>= 3.2.10'
gem.add_runtime_dependency 'multi_json', '~> 1.15'
gem.add_runtime_dependency 'link_header', '~> 0.0', '>= 0.0.8'
gem.add_runtime_dependency 'htmlentities', '~> 4.3'
gem.add_runtime_dependency 'json-canonicalization', '~> 0.3'
gem.add_runtime_dependency 'htmlentities', '~> 4.3'
gem.add_runtime_dependency 'link_header', '~> 0.0', '>= 0.0.8'
gem.add_runtime_dependency 'multi_json', '~> 1.15'
gem.add_runtime_dependency "rack", '>= 2.2', '< 4'
gem.add_development_dependency 'sinatra-linkeddata','~> 3.2'
gem.add_runtime_dependency 'rdf', '~> 3.2', '>= 3.2.10'
gem.add_development_dependency 'jsonlint', '~> 0.4' unless is_java
gem.add_development_dependency 'oj', '~> 3.14' unless is_java
gem.add_development_dependency 'yajl-ruby', '~> 1.4' unless is_java
gem.add_development_dependency 'oj', '~> 3.14' unless is_java
gem.add_development_dependency 'rack-test', '>= 1.1', '< 3'
gem.add_development_dependency 'rdf-isomorphic', '~> 3.2'
gem.add_development_dependency 'rdf-spec', '~> 3.2'
Expand All @@ -51,7 +48,9 @@ Gem::Specification.new do |gem|
gem.add_development_dependency 'rdf-xsd', '~> 3.2'
gem.add_development_dependency 'rspec', '~> 3.12'
gem.add_development_dependency 'rspec-its', '~> 1.3'
gem.add_development_dependency 'yard' , '~> 0.9'
gem.add_development_dependency 'sinatra-linkeddata', '~> 3.2'
gem.add_development_dependency 'yajl-ruby', '~> 1.4' unless is_java
gem.add_development_dependency 'yard', '~> 0.9'

gem.post_install_message = nil
gem.post_install_message = nil
end
30 changes: 26 additions & 4 deletions lib/json/ld/context.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1544,10 +1544,10 @@ def compact_iri(iri, base: nil, reverse: false, value: nil, vocab: nil)
candidates = []

term_definitions.each do |term, td|
next if td.nil? || td.id.nil? || td.id == iri || !iri.start_with?(td.id)

# Skip term if `@prefix` is not true in term definition
next unless td.prefix?
next unless td&.prefix?
Copy link
Member

@gkellogg gkellogg Apr 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Learn something new every day 😄

Copy link
Member Author

@abrisse abrisse Apr 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite practical, isn't it?


next if td&.id.nil? || td.id == iri || !td.match_iri?(iri)

suffix = iri[td.id.length..]
ciri = "#{term}:#{suffix}"
Expand All @@ -1572,7 +1572,7 @@ def compact_iri(iri, base: nil, reverse: false, value: nil, vocab: nil)

# If iri could be confused with a compact IRI using a term in this context, signal an error
term_definitions.each do |term, td|
next unless iri.to_s.start_with?("#{term}:") && td.prefix?
next unless td.prefix? && td.match_compact_iri?(iri)

raise JSON::LD::JsonLdError::IRIConfusedWithPrefix, "Absolute IRI '#{iri}' confused with prefix '#{term}'"
end
Expand Down Expand Up @@ -2204,6 +2204,22 @@ def protected?
!!@protected
end

# Returns true if the term matches a IRI
#
# @param iri [String] the IRI
# @return [Boolean]
def match_iri?(iri)
iri.start_with?(id)
end

# Returns true if the term matches a compact IRI
#
# @param iri [String] the compact IRI
# @return [Boolean]
def match_compact_iri?(iri)
iri.start_with?(prefix_colon)
end

# Set container mapping, from an array which may include @set
def container_mapping=(mapping)
mapping = case mapping
Expand Down Expand Up @@ -2328,6 +2344,12 @@ def inspect
v << "has-context" unless context.nil?
v.join(" ") + "]"
end

private

def prefix_colon
@prefix_colon ||= "#{term}:".freeze
end
end
end
end
Expand Down
4 changes: 2 additions & 2 deletions lib/json/ld/flatten.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,9 +255,9 @@ def rename_bnodes(node)
when Array
node.map { |n| rename_bnodes(n) }
when Hash
node.inject({}) do |memo, (k, v)|
node.each_with_object({}) do |(k, v), memo|
v = namer.get_name(v) if k == '@id' && v.is_a?(String) && blank_node?(v)
memo.merge(k => rename_bnodes(v))
memo[k] = rename_bnodes(v)
end
else
node
Expand Down
Loading