Skip to content

Commit

Permalink
docs: hosted documentation website (datahub-project#2174)
Browse files Browse the repository at this point in the history
  • Loading branch information
hsheth2 authored Mar 5, 2021
1 parent c479191 commit 20bf794
Show file tree
Hide file tree
Showing 67 changed files with 13,503 additions and 110 deletions.
34 changes: 34 additions & 0 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: documentation

on:
pull_request:
branches:
- master
push:
branches:
- master
# release:
# types: [published, edited]

jobs:
gh-pages:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
with:
node-version: '14.x'
- name: Build
run: |
cd docs-website
yarn install --frozen-lockfile
yarn run lint-check
yarn run generate
yarn run build
- name: Deploy
if: github.event_name == 'push'
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./docs-website/build
cname: datahubproject.io
50 changes: 42 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,13 @@
[![Docker Pulls](https://img.shields.io/docker/pulls/linkedin/datahub-gms.svg)](https://hub.docker.com/r/linkedin/datahub-gms)
[![Get on Slack](https://img.shields.io/badge/slack-join-orange.svg)](https://join.slack.com/t/datahubspace/shared_invite/zt-dkzbxfck-dzNl96vBzB06pJpbRwP6RA)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/linkedin/datahub/blob/master/docs/CONTRIBUTING.md)
[![License](https://img.shields.io/github/license/linkedin/datahub)](LICENSE)
[![GitHub commit activity](https://img.shields.io/github/commit-activity/m/linkedin/datahub)](https://github.com/linkedin/datahub/pulls?q=is%3Apr)
[![License](https://img.shields.io/github/license/linkedin/datahub)](https://github.com/linkedin/datahub/blob/master/LICENSE)

---

[Quickstart](docs/quickstart.md) |
[Documentation](#documentation) |
[Documentation](https://datahubproject.io/docs/) |
[Features](docs/features.md) |
[Roadmap](docs/roadmap.md) |
[Adoption](#adoption) |
Expand All @@ -18,7 +19,31 @@

---

<!--HOSTED_DOCS_ONLY
import ThemedImage from '@theme/ThemedImage';
import useBaseUrl from '@docusaurus/useBaseUrl';
import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
export const Logo = (props) => {
const siteConfig = useDocusaurusContext().siteConfig;
return (
<ThemedImage
alt="DataHub Logo"
sources={{
light: useBaseUrl(siteConfig.themeConfig.navbar.logo.src),
dark: useBaseUrl(siteConfig.themeConfig.navbar.logo.srcDark),
}}
{...props}
/>
)
};
<Logo style={{"maxWidth": "16em", "margin": "0 auto"}} />
<!--
HOSTED_DOCS_ONLY-->
![DataHub](docs/imgs/datahub-logo.png)
<!-- -->

> 📣 Next DataHub town hall meeting on Mar 19th, 7am-8am PDT ([convert to your local time](https://greenwichmeantime.com/time/to/pacific-local/))
> - Topic Proposals: [submit here](https://docs.google.com/forms/d/1v2ynbAXjJlqY97xE_X1DAntNrXDznOFiNfryUkMPtkI/)
Expand All @@ -41,19 +66,28 @@ DataHub is LinkedIn's generalized metadata search & discovery tool. Read about t
## Quickstart
Please follow the [DataHub Quickstart Guide](docs/quickstart.md) to get a copy of DataHub up & running locally using [Docker](https://docker.com). As the guide assumes some basic knowledge of Docker, we'd recommend you to go through the "Hello World" example of [A Docker Tutorial for Beginners](https://docker-curriculum.com) if Docker is completely foreign to you.

## Demo and Screenshots

There's a [hosted demo environment](docs/demo.md) where you can play around with DataHub before installing.

[![DataHub Demo GIF](docs/imgs/demo_large.gif)](docs/demo.md)

## Source Code and Repositories
* [linkedin/datahub](https://github.com/linkedin/datahub): This repository contains the complete source code for both DataHub's frontend & backend services. We currently follow a hybrid open source model for development in this repository. See [this blog post](https://engineering.linkedin.com/blog/2020/open-sourcing-datahub--linkedins-metadata-search-and-discovery-p) for details on how we do it.
* [linkedin/datahub-gma](https://github.com/linkedin/datahub-gma): This repository contains the source code for DataHub's metadata infrastructure libraries (Generalized Metadata Architecture, or GMA). We follow an open-source-first model for development in this repository.

## Documentation
We have documentation available at [https://datahubproject.io/docs/](https://datahubproject.io/docs/).

* [DataHub Developer's Guide](docs/developers.md)
* [DataHub Architecture](docs/architecture/architecture.md)
* [DataHub Onboarding Guide](docs/how/entity-onboarding.md)
* [Docker Images](docker)
* [Frontend](datahub-frontend)
* [Web App](datahub-web)
* [Generalized Metadata Service](gms)
* Metadata Ingestion: \[[python](metadata-ingestion)\] \[[java](metadata-ingestion-examples)\]
* [Docker Images](docker/README.md)
* [Frontend](datahub-frontend/README.md)
* Older [Ember Web App](datahub-web)
* New [React Web App](datahub-web-react/README.md) (incubating)
* [Generalized Metadata Service](gms/README.md)
* Metadata Ingestion with [Python](metadata-ingestion/README.md) (recommended) or [Java](metadata-ingestion-examples)
* [Metadata Processing Jobs](metadata-jobs)

## Releases
Expand Down Expand Up @@ -109,7 +143,7 @@ Here is a list of companies that are currently building POC or seriously evaluat
* [DataHub: A Generalized Metadata Search & Discovery Tool](https://engineering.linkedin.com/blog/2019/data-hub)
* [DataHub: Popular Metadata Architectures Explained](https://engineering.linkedin.com/blog/2020/datahub-popular-metadata-architectures-explained)
* [Open sourcing DataHub: LinkedIn’s metadata search and discovery platform](https://engineering.linkedin.com/blog/2020/open-sourcing-datahub--linkedins-metadata-search-and-discovery-p)
* [DataHub: Powering LinkedIn's Metadata](https://github.com/linkedin/datahub/blob/master/docs/demo/DataHub%20-%20Powering%20LinkedIn%E2%80%99s%20Metadata.pdf) @ [Budapest Data Forum 2020](https://budapestdata.hu/2020/en/)
* [DataHub: Powering LinkedIn's Metadata](docs/demo/DataHub_-_Powering_LinkedIn_Metadata.pdf) @ [Budapest Data Forum 2020](https://budapestdata.hu/2020/en/)
* [Taming the Data Beast Using DataHub](https://www.youtube.com/watch?v=bo4OhiPro7Y) @ [Data Engineering Melbourne Meetup November 2020](https://www.meetup.com/Data-Engineering-Melbourne/events/kgnvlrybcpbjc/)
* [Metadata Management And Integration At LinkedIn With DataHub](https://www.dataengineeringpodcast.com/datahub-metadata-management-episode-147/) @ [Data Engineering Podcast](https://www.dataengineeringpodcast.com)
* [The evolution of metadata: LinkedIn’s story](https://speakerdeck.com/shirshanka/the-evolution-of-metadata-linkedins-journey-strata-nyc-2019) @ [Strata Data Conference 2019](https://conferences.oreilly.com/strata/strata-ny-2019.html)
Expand Down
9 changes: 7 additions & 2 deletions contrib/kubernetes/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
# Kubernetes Setup for DataHub
---
title: "Deploying with Kubernetes"
hide_title: true
---

# Kubernetes Setup for DataHub

## Introduction
This directory provides the Kubernetes [Helm](https://helm.sh/) charts for DataHub.
[This directory](https://github.com/linkedin/datahub/tree/master/contrib/kubernetes/datahub) provides the Kubernetes [Helm](https://helm.sh/) charts for DataHub.

## Setup
This kubernetes deployment doesn't contain the below artifacts. The idea is to use the original helm charts for deploying each of these separately.
Expand Down
2 changes: 1 addition & 1 deletion datahub-gms-graphql-service/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Datahub GMS GraphQL Service
# DataHub GMS GraphQL Service

Datahub GMS GraphQL Service wraps the Generalized Metadata Store (GMS) Rest.li calls around a GraphQL API.

Expand Down
3 changes: 1 addition & 2 deletions datahub-web-react/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
DataHub React App (Incubating)
==============================================================================
# DataHub React App (Incubating)

## About
This module contains a React version of the DataHub UI, which is currently under incubation. Notice that this
Expand Down
5 changes: 5 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
title: "Deploying with Docker"
hide_title: true
---

# Docker Images

## Prerequisites
Expand Down
22 changes: 22 additions & 0 deletions docs-website/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/genDocs

# Dependencies
/node_modules

# Production
/build

# Generated files
.docusaurus
.cache-loader

# Misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local

npm-debug.log*
yarn-debug.log*
yarn-error.log*
33 changes: 33 additions & 0 deletions docs-website/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Website

This website is built using [Docusaurus 2](https://v2.docusaurus.io/), a modern static website generator.

## Installation

```console
yarn install
```

## Local Development

```console
yarn start
```

This command starts a local development server and open up a browser window. Most changes are reflected live without having to restart the server.

## Build

```console
yarn build
```

This command generates static content into the `build` directory and can be served using any static contents hosting service.

## Deployment

```console
GIT_USER=<Your GitHub username> USE_SSH=true yarn deploy
```

If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
3 changes: 3 additions & 0 deletions docs-website/babel.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
module.exports = {
presets: [require.resolve("@docusaurus/core/lib/babel/preset")],
};
162 changes: 162 additions & 0 deletions docs-website/docusaurus.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
module.exports = {
title: "DataHub",
tagline: "A Generalized Metadata Search & Discovery Tool",
url: "https://datahubproject.io",
baseUrl: "/",
onBrokenLinks: "throw",
onBrokenMarkdownLinks: "throw",
favicon: "img/favicon.ico",
organizationName: "linkedin", // Usually your GitHub org/user name.
projectName: "datahub", // Usually your repo name.
themeConfig: {
navbar: {
title: "DataHub",
logo: {
alt: "DataHub Logo",
src: "img/logo-color.png",
srcDark: "img/logo-dark.png",
},
items: [
{
to: "docs/",
activeBasePath: "docs",
label: "Docs",
position: "right",
},
{
to:
"https://join.slack.com/t/datahubspace/shared_invite/zt-dkzbxfck-dzNl96vBzB06pJpbRwP6RA",
label: "Slack",
position: "right",
},
{
to: "docs/demo",
label: "Demo",
position: "right",
},
{
href: "https://github.com/linkedin/datahub",
label: "GitHub",
position: "right",
},
],
},
footer: {
style: "dark",
links: [
{
title: "Docs",
items: [
{
label: "Introduction",
to: "docs/",
},
{
label: "Quickstart",
to: "docs/quickstart",
},
{
label: "Features",
to: "docs/features",
},
{
label: "FAQs",
to: "docs/faq",
},
],
},
{
title: "Community",
items: [
{
label: "Slack",
href:
"https://join.slack.com/t/datahubspace/shared_invite/zt-dkzbxfck-dzNl96vBzB06pJpbRwP6RA",
},
{
label: "YouTube",
href: "https://www.youtube.com/channel/UC3qFQC5IiwR5fvWEqi_tJ5w",
},
{
label: "Town Halls",
to: "docs/townhalls",
},
{
label: "Adoption",
to: "docs/#adoption",
},
],
},
{
title: "More",
items: [
{
label: "Demo",
href: "docs/demo",
},
{
label: "Roadmap",
to: "docs/roadmap",
},
{
label: "Contributing",
to: "docs/contributing",
},
{
label: "GitHub",
href: "https://github.com/linkedin/datahub",
},
],
},
],
copyright: `Copyright © 2015-${new Date().getFullYear()} DataHub Project Authors.`,
},
// prism: {
// theme: require('prism-react-renderer/themes/github'),
// darkTheme: require('prism-react-renderer/themes/dracula'),
// },
gtag: {
trackingID: "G-2G54RXWD4D",
},
algolia: {
apiKey: "26a4b687e96e7476b5a6f11365a83336",
indexName: "datahubproject",
// contextualSearch: true,
// searchParameters: {},
// debug: true,
},
},
presets: [
[
"@docusaurus/preset-classic",
{
docs: {
path: "genDocs",
sidebarPath: require.resolve("./sidebars.js"),
editUrl: "https://github.com/linkedin/datahub/blob/master/",
// TODO: make these work correctly with the doc generation
showLastUpdateAuthor: true,
showLastUpdateTime: true,
},
blog: false,
theme: {
customCss: require.resolve("./src/css/custom.css"),
},
},
],
],
plugins: [
"@docusaurus/plugin-ideal-image",
// '@docusaurus/plugin-google-gtag',
// [
// require.resolve("@easyops-cn/docusaurus-search-local"),
// {
// // `hashed` is recommended as long-term-cache of index file is possible.
// hashed: true,
// language: ["en"],
// docsDir: "genDocs",
// blogDir: [],
// },
// ],
],
};
Loading

0 comments on commit 20bf794

Please sign in to comment.