├── RDF-LAMP.md ├── README.md ├── docker ├── .env.dist ├── README.md ├── docker-compose.yml ├── initdb.d │ └── setup.sql └── rdf │ └── example.ttl └── src └── GetEasierRdfIssuesByCategory.py /RDF-LAMP.md: -------------------------------------------------------------------------------- 1 | # RDF-LAMP 2 | 3 | This page is for curating a list of free and open source tools and libraries -- analogous to the [LAMP stack](https://en.wikipedia.org/wiki/LAMP_%28software_bundle%29) -- that a new RDF user can easily download and use to build a "typical" RDF application. It addresses [issue #2](https://github.com/w3c/EasierRDF/issues/2). This list is not intended to be exhaustive. (See [Awesome Semantic Web](https://github.com/semantalytics/awesome-semantic-web) for a more comprehensive list.) Rather it is intended to be a starting point: to include only those tools and libraries that would be needed by _most_ RDF applications. The hope is to eventually bundle them into a single, common download, analogous to Red Hat or Ubuntu. 4 | 5 | _**PRs are invited!**_ 6 | 7 | ## Target applications 8 | Three "typical" RDF applications are targeted: 9 | 10 | * **Data integration.** The application must integrate data from multiple data sources having different formats and data models, including a relational database and a (plain) JSON data source. 11 | * **Catalog metadata.** The application must use RDF metadata to categorize items in multiple ways, such as products in a catalog. 12 | * **OWL for smarter querying.** The application must use OWL reasoning (or other inference rules) to enable smarter queries, such as queries of biomedical data. 13 | 14 | One or more of these applications should involve the following challenges: 15 | 16 | * Alignment of multiple data models / vocabularies / ontologies. 17 | * Use of user-defined inference, either using an RDF rules language or whatever programming language is used to build the application. 18 | * Conversion of relational data to RDF. 19 | * Conversion of plain JSON data -- not already JSON-LD -- to RDF. 20 | * Using public RDF data. 21 | * Understanding and visualizing RDF data. 22 | * SPARQL queries. 23 | * Use of SKOS for metadata. 24 | * Metadata vocabulary that originates in a spreadsheet. 25 | * Conversion of RDF data: 26 | * from RDF/XML or Turtle to Turtle or N-Triples; 27 | * from JSON-LD or TriG to N-Quads; 28 | * from N-Quads to human-friendly TriG, using common RDF namespace prefixes. 29 | * (What else?) 30 | 31 | **TO DO:** _Find or create specific examples for the above three kinds of application, with example data, and turn them into tutorials based on the below set of tools. (Can someone suggest some, complete with example data? Or suggest others, perhaps some existing tutorials?)_ 32 | 33 | ## Criteria for inclusion 34 | To be considered for inclusion, a candidate tool or library: 35 | * must be [OSI-compliant free and open source software (FOSS)](https://opensource.org/), though the software may be maintained by a commercial company and it may also be offered under a non-FOSS license; 36 | * must be compliant with W3C standards, especially RDF 1.1, SPARQL 1.1 and JSON-LD 1.1; 37 | * must be necessary (or normally recommended) to implement at least one of the three "typical" applications listed above; 38 | * must be powerful enough to be used for real world applications (in its FOSS version); 39 | * must be actively supported and used in the RDF community; 40 | * should run on Linux, Windows and Mac OS; and 41 | * should represent the easiest and most popular community choice in its category. 42 | * should represent the easiest within its tool category, to get started and use (i.e., usability, documentation, active community forum, stability, etc.) 43 | 44 | ## Candidate Tools 45 | Tools in this section are not specific to the programming language that is used to build your RDF application. 46 | 47 | * One of: 48 | * [Virtuoso Open Source](https://github.com/openlink/virtuoso-opensource/) -- RDF database 49 | * [RDF4J](https://www.rdf4j.org/)-- RDF database 50 | * [Apache Jena](https://jena.apache.org/)-- RDF database 51 | * [Oxigraph](https://github.com/oxigraph/oxigraph)-- RDF database 52 | * [Protege](https://protege.stanford.edu) -- Ontology editor 53 | * One of: 54 | * [pySHACL](https://github.com/RDFLib/pySHACL/) -- Validate RDF against a data schema 55 | * ShEx -- Validate RDF against a data schema TODO: Add correct URL 56 | * [rapper -- Raptor RDF parsing and serializing utility](https://librdf.org/raptor/rapper.html) -- Validates and converts different RDF serialization formats 57 | * [SKOS Play](https://skos-play.sparna.fr/play/about)? -- Converts a spreadsheet taxonomy to SKOS data 58 | * [YASGUI](https://triply.cc/docs/yasgui) -- SPARQL editor 59 | * [OpenRefine](https://openrefine.org/) -- Clean data and convert to RDF 60 | * One of: 61 | * [OpenLink Structured Data Sniffer](https://github.com/OpenLinkSoftware/OSDS_extension) -- View RDF in web pages 62 | * [RDF Browser](https://addons.mozilla.org/en-US/firefox/addon/rdf-browser/) -- View RDF in web pages 63 | * (What for writing and applying custom inference rules?) 64 | * (What to convert from other formats to RDF?) 65 | * (What for RDF authoring?) 66 | * (What for visualizing RDF data?) 67 | * (What for converting from relational data to triples?) 68 | * (What for converting from JSON data to triples (assuming it is not already JSON-LD)?) 69 | * (What for writing and applying simple inference rules) 70 | * (list others here ... ) 71 | 72 | ## Candidate libraries 73 | The programming language you use to build your RDF application will determine which libraries are applicable to you. 74 | 75 | ### Java libraries 76 | 77 | * [Apache Jena](https://jena.apache.org/download/index.cgi) -- Java framework and library for building RDF applications 78 | * (list others here ... ) 79 | 80 | ### JavaScript libraries 81 | 82 | * [RDF JS](https://rdf.js.org/) -- JavaScript library for building RDF applications. 83 | * (list others here ... ) 84 | 85 | ### Python libraries 86 | 87 | * [RDFLib](https://pypi.org/project/rdflib/) -- Python library for RDF applications. 88 | * (list others here ... ) 89 | 90 | For a curated list of additional projects specifically for Python, see [semantic-python-overview](https://github.com/pysemtec/semantic-python-overview). 91 | 92 | ### Ruby libraries 93 | 94 | * [Ruby-RDF](https://github.com/ruby-rdf/linkeddata) 95 | 96 | ### Rust libraries 97 | 98 | * [Sophia](https://github.com/pchampin/sophia_rs) -- "Linked Data and Semantic Web toolkit", provides in-memory graphs for efficient triple pattern queries. 99 | * [Oxigraph](https://github.com/oxigraph/oxigraph)-- On-disk graph database library. 100 | * [Rio](https://crates.io/crates/rio_turtle/0.6.2) -- Low level Turtle, Trig, N-Triples and N-Quads parsers and serializers. 101 | 102 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # EasierRDF 2 | 3 | _This repository is for experimental/exploratory work on making RDF easier to use, with the goal of making it easy enough for most developers. By "RDF" we mean the whole RDF ecosystem -- including SPARQL, OWL, tools, standards, educational materials, etc. -- everything that a developer touches when using RDF. Our plan:_ 4 | * _focus and coordinate community efforts;_ 5 | * _build on existing standards whenever possible;_ 6 | * _launch additional W3C Community Groups to tackle specific areas of need, including_ 7 | * _[n3-dev Community Group](https://www.w3.org/community/n3-dev/), for standardizing N3 rules; and_ 8 | * _contribute to (and benefit from) related efforts, such as the [W3C Graph Data Workshop](https://www.w3.org/Data/events/data-ws-2019/cfp.html) in March 2019, which prepare the path for launching new standards work at W3C._ 9 | 10 | ## Introduction 11 | 12 | The value of RDF for graph data has been well proven, in many applications, over the 20+ years since it was first created. However, difficulty of use has caused RDF to be categorized as a niche technology. This is unfortunate because it limits uptake and prevents RDF from being viewed as a viable choice for many use cases that would otherwise be an excellent fit. 13 | 14 | This work seeks to build upon our experience with RDF to examine how we can make it easier to use. What aspects or gaps have caused difficulty? How can RDF better support features that users commonly need and other graph databases offer? How can we make RDF -- or a successor -- easy enough for most developers? 15 | 16 | At the same time, businesses are now showing a rapidly growing interest in graph data. Businesses have used relational databases for many years, but it is costly to adapt database schema and applications in response to evolving application needs. Other graph and NoSQL databases have emerged to help meet this need. Unfortunately, there is a lack of interoperability across existing graph data solutions, motivating [interest in open standards for an interchange framework](https://www.w3.org/Data/events/data-ws-2019/cfp.html). RDF is an appealing vendor neutral framework for graph data, and is well positioned to take on the role of an interchange framework. Although this interest in RDF as a graph interchange framework arose independently from the effort to make RDF easier, and has different goals, there is a natural overlap in motivation, and both efforts can benefit each other. 17 | 18 | ## Guiding principles 19 | 20 | **1. The goal is to make RDF -- or some RDF-based successor -- easy enough for most developers, who are new to RDF, to be consistently successful.** 21 | 22 | **2. Solutions may involve anything in the RDF ecosystem: standards, tools, guidance, etc. All options are on the table.** 23 | 24 | **3. Backward compatibility is highly desirable, but *less* important than ease of use.** 25 | 26 | ## How to contribute? 27 | 28 | We welcome contributions. A good place to start is to review the [issues list](https://github.com/w3c/EasierRDF/issues), categorized below. Please feel free to start a new issue if none of the existing ones are a good match. You can also send comments to the mailing list: [semantic-web@w3.org](https://lists.w3.org/Archives/Public/semantic-web/). 29 | 30 | ## Intellectual property rights 31 | 32 | This work is being performed under the [W3C rdf-dev Community Group](https://www.w3.org/community/rdf-dev/) and is subject to the [W3C Community Contributor License Agreement (CLA)](https://www.w3.org/community/about/agreements/cla/). 33 | 34 | # Issue topics 35 | 36 | Issues and ideas are recorded in our [issues list](https://github.com/w3c/EasierRDF/issues) and divided into the categories below using [issue labels](https://github.com/w3c/EasierRDF/labels). The lists below are *not* auto-populated, so click on the category name below to see the latest list. A [script is available](https://github.com/w3c/EasierRDF/blob/master/src/GetEasierRdfIssuesByCategory.py) for regenerating this list, but it currently must be run manually. 37 | 38 | ## [Category: big ideas](https://github.com/w3c/EasierRDF/labels/Category%3A%20big%20ideas): For major ideas that span multiple issue categories 39 | 40 | * [Issue 68: Idea: Record stronger alternative definitions of ease](https://github.com/w3c/EasierRDF/issues/68) 41 | * [Issue 34: Idea: Higher-level RDF language](https://github.com/w3c/EasierRDF/issues/34) 42 | * [Issue 33: Idea: Build powerful query engines](https://github.com/w3c/EasierRDF/issues/33) 43 | * [Issue 32: Idea: Build JavaScript libraries that work in the browser](https://github.com/w3c/EasierRDF/issues/32) 44 | * [Issue 31: Idea: Base Linked Data on a JavaScript API](https://github.com/w3c/EasierRDF/issues/31) 45 | * [Issue 30: Idea: Attract JavaScript front-end developers](https://github.com/w3c/EasierRDF/issues/30) 46 | 47 | 48 | ## [Category: tools](https://github.com/w3c/EasierRDF/labels/Category%3A%20tools): For RDF tools 49 | 50 | * [Issue 53: Lack of good forms-based end-user editor](https://github.com/w3c/EasierRDF/issues/53) 51 | * [Issue 52: Transparently merge rdf: and rdfs: namespaces](https://github.com/w3c/EasierRDF/issues/52) 52 | * [Issue 43: Nobody knows who does what with what](https://github.com/w3c/EasierRDF/issues/43) 53 | * [Issue 40: Spoggy easy create & read graphs](https://github.com/w3c/EasierRDF/issues/40) 54 | * [Issue 39: SPARQL Triplestore and Reasoning Performance](https://github.com/w3c/EasierRDF/issues/39) 55 | * [Issue 38: Lack of Full OWL2 Support in Triplestores](https://github.com/w3c/EasierRDF/issues/38) 56 | * [Issue 37: Lack of RDF Visualisation Software](https://github.com/w3c/EasierRDF/issues/37) 57 | * [Issue 35: Lack of a Good Editor](https://github.com/w3c/EasierRDF/issues/35) 58 | * [Issue 33: Idea: Build powerful query engines](https://github.com/w3c/EasierRDF/issues/33) 59 | * [Issue 5: Moribundity of Tools](https://github.com/w3c/EasierRDF/issues/5) 60 | * [Issue 4: Overview of an RDF triple store](https://github.com/w3c/EasierRDF/issues/4) 61 | * [Issue 3: Lack of automated feedback / validation](https://github.com/w3c/EasierRDF/issues/3) 62 | * [Issue 2: Tools are scattered](https://github.com/w3c/EasierRDF/issues/2) 63 | 64 | 65 | ## [Category: education](https://github.com/w3c/EasierRDF/labels/Category%3A%20education): For documentation and education 66 | 67 | * [Issue 59: Reduce the jargon](https://github.com/w3c/EasierRDF/issues/59) 68 | * [Issue 58: Confusion between RDF and Linked Data](https://github.com/w3c/EasierRDF/issues/58) 69 | * [Issue 49: Popular use cases for graph data](https://github.com/w3c/EasierRDF/issues/49) 70 | * [Issue 41: Idea: Material to be put in an undergraduate DB or WebDev unit](https://github.com/w3c/EasierRDF/issues/41) 71 | * [Issue 11: Rejection of SQL and OO as Metaphors](https://github.com/w3c/EasierRDF/issues/11) 72 | * [Issue 10: Lack of a Canonical Example](https://github.com/w3c/EasierRDF/issues/10) 73 | * [Issue 9: Lack of Technology Framing](https://github.com/w3c/EasierRDF/issues/9) 74 | * [Issue 8: Beginner friendly support](https://github.com/w3c/EasierRDF/issues/8) 75 | * [Issue 7: Beginner friendly tutorials / documentation](https://github.com/w3c/EasierRDF/issues/7) 76 | * [Issue 6: No clear entry point](https://github.com/w3c/EasierRDF/issues/6) 77 | 78 | 79 | ## [Category: usage](https://github.com/w3c/EasierRDF/labels/Category%3A%20usage): For issues around RDF usage in practice 80 | 81 | * [Issue 66: Dereferencing RDF, RDFS & OWL in a browser should return human-friendly versions](https://github.com/w3c/EasierRDF/issues/66) 82 | * [Issue 65: Do not use anonymous classes](https://github.com/w3c/EasierRDF/issues/65) 83 | * [Issue 43: Nobody knows who does what with what](https://github.com/w3c/EasierRDF/issues/43) 84 | * [Issue 16: Easier-to-use competitors](https://github.com/w3c/EasierRDF/issues/16) 85 | * [Issue 15: Define an easier profile of RDF](https://github.com/w3c/EasierRDF/issues/15) 86 | * [Issue 14: Namespaces are lost in the RDF model](https://github.com/w3c/EasierRDF/issues/14) 87 | * [Issue 13: Namespace proliferation](https://github.com/w3c/EasierRDF/issues/13) 88 | * [Issue 12: IRI allocation](https://github.com/w3c/EasierRDF/issues/12) 89 | 90 | 91 | ## [Category: language features](https://github.com/w3c/EasierRDF/labels/Category%3A%20language%20features): For language features of RDF itself -- model and syntax 92 | 93 | * [Issue 62: Retain relative URIs at the RDF model level](https://github.com/w3c/EasierRDF/issues/62) 94 | * [Issue 61: Simplified Reification](https://github.com/w3c/EasierRDF/issues/61) 95 | * [Issue 57: Composition of named graphs](https://github.com/w3c/EasierRDF/issues/57) 96 | * [Issue 52: Transparently merge rdf: and rdfs: namespaces](https://github.com/w3c/EasierRDF/issues/52) 97 | * [Issue 51: Relationship to the Web of Things](https://github.com/w3c/EasierRDF/issues/51) 98 | * [Issue 48: Different kinds of identifiers](https://github.com/w3c/EasierRDF/issues/48) 99 | * [Issue 45: Property Graphs](https://github.com/w3c/EasierRDF/issues/45) 100 | * [Issue 42: Maybe "datatype" should be in the RDF Graph explicitly](https://github.com/w3c/EasierRDF/issues/42) 101 | * [Issue 25: Vocabulary for new semantic extensions](https://github.com/w3c/EasierRDF/issues/25) 102 | * [Issue 24: Decouple semantics from data model](https://github.com/w3c/EasierRDF/issues/24) 103 | * [Issue 22: Language-tagged strings](https://github.com/w3c/EasierRDF/issues/22) 104 | * [Issue 21: Literals as subjects](https://github.com/w3c/EasierRDF/issues/21) 105 | * [Issue 20: Standardized n-ary relations (and property graphs)](https://github.com/w3c/EasierRDF/issues/20) 106 | * [Issue 19: Blank nodes](https://github.com/w3c/EasierRDF/issues/19) 107 | * [Issue 17: IRI reuse and synonyms](https://github.com/w3c/EasierRDF/issues/17) 108 | 109 | 110 | ## [Category: related standards](https://github.com/w3c/EasierRDF/labels/Category%3A%20related%20standards): For RDF-related standards 111 | 112 | * [Issue 64: Mappings between shape languages](https://github.com/w3c/EasierRDF/issues/64) 113 | * [Issue 60: transparently switch between CWA and OWA](https://github.com/w3c/EasierRDF/issues/60) 114 | * [Issue 29: No standard way to map a JSON predicate to a URI](https://github.com/w3c/EasierRDF/issues/29) 115 | * [Issue 27: Lack of a standard rules language](https://github.com/w3c/EasierRDF/issues/27) 116 | * [Issue 26: Lack of standard RDF canonicalization](https://github.com/w3c/EasierRDF/issues/26) 117 | 118 | 119 | 120 | ## [Category: easier profile](https://github.com/w3c/EasierRDF/labels/Category%3A%20easier%20profile): For candidate features of an easier profile of RDF 121 | 122 | * [Issue 56: OWL is too hard](https://github.com/w3c/EasierRDF/issues/56) 123 | * [Issue 50: Profile: Exactly one type per representation of a resource](https://github.com/w3c/EasierRDF/issues/50) 124 | 125 | 126 | 127 | ## [Category: experiments](https://github.com/w3c/EasierRDF/labels/Category%3A%20experiments): For demonstrations exploring the design space 128 | 129 | * [Issue 71: Chunks and Chunk Rules](https://github.com/w3c/EasierRDF/issues/71) 130 | * [Issue 70: Graph traversal automata for RDF queries](https://github.com/w3c/EasierRDF/issues/70) 131 | * [Issue 69: Pattern based queries](https://github.com/w3c/EasierRDF/issues/69) 132 | 133 | 134 | ## Uncategorized 135 | [Search](https://github.com/w3c/EasierRDF/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+no%3Alabel) for issues with no category label. 136 | 137 | -------------------------------------------------------------------------------- /docker/.env.dist: -------------------------------------------------------------------------------- 1 | DBA_PASSWORD=changeme 2 | -------------------------------------------------------------------------------- /docker/README.md: -------------------------------------------------------------------------------- 1 | Simple template for a Docker Compose setup based on [Virtuoso](https://github.com/openlink/virtuoso-opensource/), providing an RDF store, SPARQL Endpoint, and RDF browser (among other available features) for a quick knowledge base deployment. 2 | 3 | # Localize 4 | 5 | 1. Replace `rdf/example.ttl` with your knowledge base 6 | 2. Add your namespaces to `initdb.d/setup.sql` 7 | 3. Adapt `docker-compose.yml` 8 | 4. Copy `.env.dist` to `.env` and change `DBA_PASSWORD` 9 | 10 | # Start to use 11 | 12 | 1. Run `docker compose up --build` 13 | 2. Test the SPARQL endpoint at [`http://localhost:8890/sparql`](http://localhost:8890/sparql) 14 | 2. Test the RDF browser at [`http://localhost:8080/ExInstance`](http://localhost:8080/ExInstance) 15 | -------------------------------------------------------------------------------- /docker/docker-compose.yml: -------------------------------------------------------------------------------- 1 | services: 2 | 3 | virtuoso: 4 | image: openlink/virtuoso-opensource-7 5 | environment: 6 | - DBA_PASSWORD=${DBA_PASSWORD} 7 | - VIRT_DATABASE_ERRORLOGLEVEL=3 8 | - VIRT_SPARQL_DEFAULTGRAPH=http://example.org 9 | - VIRT_SPARQL_DEFAULTQUERY=select distinct * {?s ?p ?o.} LIMIT 100 10 | - VIRT_PARAMETERS_DIRSALLOWED=., /usr/local/virtuoso-opensource/share/virtuoso/vad, /rdf 11 | - VIRT_PLUGINS_-=- 12 | volumes: 13 | - ./rdf:/rdf:ro 14 | - ./initdb.d:/opt/virtuoso-opensource/initdb.d:ro 15 | ports: 16 | - "127.0.0.1:8890:8890" 17 | restart: unless-stopped 18 | 19 | lodview: 20 | image: ghcr.io/konradhoeffner/lodview:22.05 21 | environment: 22 | - LodViewendpoint=http://virtuoso:8890/sparql 23 | - LodViewIRInamespace=http://example.org/ 24 | - LodViewhomeUrl=/ExInstance 25 | ports: 26 | - "127.0.0.1:8080:8080" 27 | depends_on: 28 | - virtuoso 29 | restart: unless-stopped 30 | -------------------------------------------------------------------------------- /docker/initdb.d/setup.sql: -------------------------------------------------------------------------------- 1 | log_message('Setup: Activate CORS'); 2 | update DB.DBA.HTTP_PATH set HP_OPTIONS = serialize(vector('browse_sheet', '', 'noinherit', 'yes', 'cors', '*', 'cors_restricted', 0)) where HP_LPATH = '/sparql'; 3 | log_message('Setup: Declare namespaces'); 4 | DB.DBA.XML_SET_NS_DECL ('ex', 'http://example.org/', 2); 5 | log_message('Setup: Load data'); 6 | ld_dir_all ('/rdf/', '*.ttl', 'http://example.org'); 7 | rdf_loader_run(); 8 | log_message('Setup: Finished'); 9 | 10 | -------------------------------------------------------------------------------- /docker/rdf/example.ttl: -------------------------------------------------------------------------------- 1 | PREFIX rdfs: 2 | PREFIX owl: 3 | PREFIX ex: 4 | 5 | ex:ExClass 6 | a owl:Class ; 7 | rdfs:comment "an example class"@en ; 8 | rdfs:label "example class"@en . 9 | 10 | ex:ExInstance 11 | a ex:ExClass ; 12 | ex:exProperty 5 ; 13 | rdfs:comment "an example instance."@en ; 14 | rdfs:label "example instance"@en . 15 | 16 | ex:exProperty 17 | a owl:DatatypeProperty ; 18 | rdfs:domain ex:ExClass ; 19 | rdfs:label "Beispielproperty"@de , 20 | "example property"@en . 21 | -------------------------------------------------------------------------------- /src/GetEasierRdfIssuesByCategory.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | # coding: utf-8 4 | 5 | # In[26]: 6 | 7 | 8 | '''This is for updating the issues lists in the EasierRDF github repo at 9 | https://github.com/w3c/EasierRDF . 10 | 11 | Usage: 12 | ./GetEasierRdfIssuesByCategory.py > README.md 13 | 14 | Author: David Booth 11/5/21 15 | License: CC0 16 | ''' 17 | 18 | import urllib.request 19 | import urllib.parse 20 | import json 21 | import re 22 | 23 | issuesUrl='https://github.com/w3c/EasierRDF/issues' 24 | categoriesUrl='https://github.com/w3c/EasierRDF/labels?q=Category%3A' 25 | readmeUrl = 'https://raw.githubusercontent.com/w3c/EasierRDF/master/README.md' 26 | 27 | def GetCategories(categoriesUrl='https://github.com/w3c/EasierRDF/labels?q=Category%3A'): 28 | '''See https://stackoverflow.com/questions/41753769/get-all-github-issues-from-repo-with-specific-label''' 29 | url = categoriesUrl 30 | url = re.sub(r'^https://github\.com/', 'https://api.github.com/repos/', url) 31 | # print(url) 32 | header={'Accept': 'application/json'} 33 | req = urllib.request.Request(url=url, headers=header, method='GET') 34 | categoriesJson = urllib.request.urlopen(req).read() 35 | categoriesObjects = json.loads(categoriesJson.decode("utf-8")) 36 | return categoriesObjects 37 | 38 | categoryNames = ['Category: big ideas', 39 | 'Category: tools', 40 | 'Category: education', 41 | 'Category: usage', 42 | 'Category: easier profile', 43 | 'Category: language features', 44 | 'Category: related standards'] 45 | 46 | categoryNames = [ co['name'] for co in GetCategories() if re.match(r"Category:", co['name']) ] 47 | # print("\n".join(categoryNames)) 48 | # exit(0) 49 | # print(json.dumps(GetCategories())) 50 | # exit(0) 51 | 52 | def GetIssuesByLabel(label='Category: big ideas', issuesUrl='https://github.com/w3c/EasierRDF/issues'): 53 | '''See https://stackoverflow.com/questions/41753769/get-all-github-issues-from-repo-with-specific-label''' 54 | url = issuesUrl + "?labels=" + urllib.parse.quote(label) 55 | url = re.sub(r'^https://github\.com/', 'https://api.github.com/repos/', url) 56 | # print(url) 57 | header={'Accept': 'application/json'} 58 | req = urllib.request.Request(url=url, headers=header, method='GET') 59 | issuesJson = urllib.request.urlopen(req).read() 60 | issuesObjects = json.loads(issuesJson.decode("utf-8")) 61 | return issuesObjects 62 | 63 | def ObjectToMarkdown(issuesObjects): 64 | issuesMdList = [ '* [Issue {}: {}]({})\n'.format(i['number'], i['title'], i['html_url']) for i in issuesObjects] 65 | issuesMd = "".join(issuesMdList) 66 | return issuesMd 67 | 68 | GetIssuesByLabel() 69 | 70 | 71 | # In[27]: 72 | 73 | 74 | def UpdateIssuesList(categoryName, page, issuesUrl='https://github.com/w3c/EasierRDF/issues'): 75 | '''Update the issue list in the given category in the page. 76 | The rest of the page is unchanged. 77 | The issue list in each category is marked like this: 78 | 79 | ## [Category: big ideas](https://github.com/w3c/EasierRDF/labels/Category%3A%20big%20ideas): For major ideas that span multiple issue categories 80 | 81 | * [Issue 34: Idea: Higher-level RDF language](https://github.com/w3c/EasierRDF/issues/34) 82 | * [Issue 33: Idea: Build powerful query engines](https://github.com/w3c/EasierRDF/issues/33) 83 | * [Issue 32: Idea: Build JavaScript libraries that work in the browser](https://github.com/w3c/EasierRDF/issues/32) 84 | * [Issue 31: Idea: Base Linked Data on a JavaScript API](https://github.com/w3c/EasierRDF/issues/31) 85 | * [Issue 30: Idea: Attract JavaScript front-end developers](https://github.com/w3c/EasierRDF/issues/30) 86 | ''' 87 | categoryEncoded = urllib.parse.quote_plus(categoryName) 88 | beginMarker = '' 89 | endMarker = '' 90 | beforeMiddleAfter = page.split(beginMarker) 91 | if len(beforeMiddleAfter) != 2: 92 | raise Exception('ERROR: Did not find beginMarker in page: '+beginMarker) 93 | before = beforeMiddleAfter[0] 94 | middleAfter = beforeMiddleAfter[1].split(endMarker) 95 | if len(middleAfter) != 2: 96 | raise Exception('ERROR: Did not find endMarker in page: '+endMarker) 97 | middle = middleAfter[0] 98 | after = middleAfter[1] 99 | issuesList = ObjectToMarkdown(GetIssuesByLabel(categoryName)) 100 | page = before + beginMarker + '\n' + issuesList + endMarker + after 101 | return page 102 | 103 | 104 | # print(ObjectToMarkdown(GetIssuesByLabel('Category: big ideas'))) 105 | 106 | 107 | 108 | # In[33]: 109 | 110 | 111 | page = ''' ## [Category: big ideas](https://github.com/w3c/EasierRDF/labels/Category%3A%20big%20ideas): For major ideas that span multiple issue categories 112 | 113 | * [Issue 31: Idea: Base Linked Data on a JavaScript API](https://github.com/w3c/EasierRDF/issues/31) 114 | * [Issue 30: Idea: Attract JavaScript front-end developers](https://github.com/w3c/EasierRDF/issues/30) 115 | ''' 116 | 117 | # print(UpdateIssuesList('Category: big ideas', page, issuesUrl)) 118 | 119 | 120 | # In[34]: 121 | 122 | def main(): 123 | page = urllib.request.urlopen(readmeUrl).read().decode("utf8") 124 | for categoryName in categoryNames: 125 | page = UpdateIssuesList(categoryName, page, issuesUrl) 126 | print(page) 127 | 128 | if __name__ == "__main__": 129 | main() 130 | 131 | --------------------------------------------------------------------------------