Skip to content

Feature/network graph #597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 18 commits into
base: main
Choose a base branch
from
Draft

Feature/network graph #597

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
541b5fe
Create a network graph component with reactflow and dagre
aswallace Oct 1, 2025
585678c
Set up database service to be able to process provenance source
aswallace Oct 1, 2025
cec133c
Install reactflow and dagre
aswallace Oct 1, 2025
ac942af
Create provenance state branch
aswallace Oct 1, 2025
7869372
Add a full-screen modal for the network graph
aswallace Oct 1, 2025
e384897
Modify the data source modal to upload provenance data
aswallace Oct 1, 2025
8e1d536
Update description for provenance modal
aswallace Oct 2, 2025
25f2312
Include provenance source in search params
aswallace Oct 7, 2025
8201b51
Fix broken searchparams unit tests
aswallace Oct 7, 2025
964f165
Add comments to custom graph edge
aswallace Oct 7, 2025
45315a6
Remove unnecessary undefined row check
aswallace Oct 7, 2025
79303c5
Make full screen modal take up more vertical space
aswallace Oct 7, 2025
65e3885
Use uid instead of optional id
aswallace Oct 9, 2025
3941d4f
Move graph construction step to load later
aswallace Oct 15, 2025
673320f
Merge branch 'main' of github.com:AllenInstitute/biofile-finder into …
aswallace Oct 21, 2025
4d0de23
Add custom file nodes
aswallace Oct 29, 2025
2adb818
Allow markdown rendering in graph edges
aswallace Oct 29, 2025
7836f08
Merge branch 'main' of github.com:AllenInstitute/biofile-finder into …
aswallace Oct 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
440 changes: 440 additions & 0 deletions package-lock.json
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,10 @@
"dependencies": {
"@aics/frontend-insights": "0.2.x",
"@aics/redux-utils": "0.6.0",
"@dagrejs/dagre": "^1.1.5",
"@fluentui/react": "8.67",
"@tippyjs/react": "4.2.x",
"@xyflow/react": "^12.8.5",
"amazon-s3-url": "^1.0.3",
"axios": "0.21.x",
"classnames": "2.2.x",
Expand All @@ -82,6 +84,7 @@
"jszip": "^3.10.1",
"lodash": "4.17.x",
"lru-cache": "5.1.x",
"markdown-to-jsx": "^8.0.0",
"normalize.css": "8.0.x",
"ome-zarr.js": "^0.0.15",
"react": "17.x",
Expand Down
224 changes: 120 additions & 104 deletions packages/core/components/DataSourcePrompt/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@ const ADDITIONAL_COLUMN_DETAILS = [
'If an "Uploaded" column is present, it should contain the date the file was uploaded to the storage and be formatted as YYYY-MM-DD HH:MM:SS.Z where Z is a timezone offset. ',
];

export enum DataSourceType {
default = 0,
metadata = 1,
provenance = 2,
Copy link
Contributor Author

@aswallace aswallace Oct 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this in for now so that I can re-use the data source prompt to upload just a provenance file on its own

}

/**
* Dialog meant to prompt user to select a data source option
*/
Expand All @@ -31,7 +37,8 @@ export default function DataSourcePrompt(props: Props) {

const selectedDataSources = useSelector(selection.selectors.getSelectedDataSources);
const dataSourceInfo = useSelector(interaction.selectors.getDataSourceInfoForVisibleModal);
const { query } = dataSourceInfo || ({} as DataSourcePromptInfo);
const { query, sourceType = DataSourceType.default } =
dataSourceInfo || ({} as DataSourcePromptInfo);
const requiresDataSourceReload = useSelector(selection.selectors.getRequiresDataSourceReload);

const [dataSource, setDataSource] = React.useState<Source>();
Expand All @@ -44,6 +51,13 @@ export default function DataSourcePrompt(props: Props) {
};

const onSubmit = (dataSource: Source, metadataSource?: Source) => {
if (sourceType === DataSourceType.provenance) {
if (dataSource) {
dispatch(selection.actions.changeSourceProvenance(dataSource));
}
// To do: include provenance source in query as with metadatasource
return onDismiss();
}
if (requiresDataSourceReload || query) {
if (metadataSource) {
dispatch(selection.actions.changeSourceMetadata(metadataSource));
Expand Down Expand Up @@ -119,130 +133,132 @@ export default function DataSourcePrompt(props: Props) {
parentId={`file-prompt-${props.isModal ? "modal" : "main"}`}
lightBackground={props.isModal}
/>
{showAdvancedOptions ? (
advancedOptions
) : (
<LinkLikeButton
className={styles.advancedOptionsButton}
onClick={() => setShowAdvancedOptions(!showAdvancedOptions)}
text="Add metadata descriptor file (optional)"
/>
)}
{showAdvancedOptions
? advancedOptions
: sourceType === DataSourceType.default && (
<LinkLikeButton
className={styles.advancedOptionsButton}
onClick={() => setShowAdvancedOptions(!showAdvancedOptions)}
text="Add metadata descriptor file (optional)"
/>
)}
<div className={styles.loadButtonContainer}>
<PrimaryButton
className={classNames(styles.loadButton)}
disabled={!dataSource}
disabled={!dataSource && !metadataSource}
text="LOAD"
onClick={() => dataSource && onSubmit(dataSource, metadataSource)}
/>
</div>
</div>
<div className={styles.guidance}>
<hr className={styles.divider} />
<h4 className={styles.subheader}>Getting started guidance and example CSV</h4>
<table
className={classNames(
styles.tableExample,
props?.isModal ? styles.darkHeader : styles.lightBorder
)}
>
<thead>
<tr>
<th>
File Path <i>(required metadata key)</i>
</th>
<th>
Gene <i>(example metadata key)</i>
</th>
<th>
Color <i>(example metadata key)</i>
</th>
</tr>
</thead>
<tbody>
<tr>
<td>/folder/folder/my_storage/filename.zarr</td>
<td>CDH2</td>
<td>Blue</td>
</tr>
<tr>
<td>/folder/my_storage/filename.txt</td>
<td>VIM</td>
<td>Green</td>
</tr>
</tbody>
</table>
<h4 className={styles.subheader}>Minimum requirements</h4>
<ul className={styles.detailsList}>
<li>
The first row should contain metadata keys (i.e., column headers), with
&quot;File Path&quot; being the only required key.
</li>
<li>
Each subsequent row should contain the values of corresponding keys for each
file.
</li>
</ul>
{sourceType === DataSourceType.default && (
<div className={styles.guidance}>
<hr className={styles.divider} />
<h4 className={styles.subheader}>Getting started guidance and example CSV</h4>
<table
className={classNames(
styles.tableExample,
props?.isModal ? styles.darkHeader : styles.lightBorder
)}
>
<thead>
<tr>
<th>
File Path <i>(required metadata key)</i>
</th>
<th>
Gene <i>(example metadata key)</i>
</th>
<th>
Color <i>(example metadata key)</i>
</th>
</tr>
</thead>
<tbody>
<tr>
<td>/folder/folder/my_storage/filename.zarr</td>
<td>CDH2</td>
<td>Blue</td>
</tr>
<tr>
<td>/folder/my_storage/filename.txt</td>
<td>VIM</td>
<td>Green</td>
</tr>
</tbody>
</table>
<h4 className={styles.subheader}>Minimum requirements</h4>
<ul className={styles.detailsList}>
<li>
The first row should contain metadata keys (i.e., column headers), with
&quot;File Path&quot; being the only required key.
</li>
<li>
Each subsequent row should contain the values of corresponding keys for
each file.
</li>
</ul>

{isDataSourceDetailExpanded ? (
<>
<h4 className={styles.subheader}>Advanced:</h4>
<ul className={styles.detailsList}>
<li>
Data source files can be generated by this application by selecting
some files, right-clicking, and selecting one of the &quot;Save
metadata as&quot; options.
</li>
<li>
The following are optional pre-defined columns that are handled as
special cases:
</li>
{isDataSourceDetailExpanded ? (
<>
<h4 className={styles.subheader}>Advanced:</h4>
<ul className={styles.detailsList}>
{ADDITIONAL_COLUMN_DETAILS.map((text) => (
<li key={text} className={styles.details}>
{text}
</li>
))}
<li>
Data source files can be generated by this application by
selecting some files, right-clicking, and selecting one of the
&quot;Save metadata as&quot; options.
</li>
<li>
The following are optional pre-defined columns that are handled
as special cases:
</li>
<ul className={styles.detailsList}>
{ADDITIONAL_COLUMN_DETAILS.map((text) => (
<li key={text} className={styles.details}>
{text}
</li>
))}
</ul>
<li className={styles.details}>
Optionally, you can supply an additional metadata descriptor
file to add more information about the data source. This file
should have a header row column named &quot;Column Name&quot;
and another column named &quot;Description&quot;. Each
subsequent row should contain the details for any columns
present in the actual data source you would like to describe.
</li>
</ul>
<li className={styles.details}>
Optionally, you can supply an additional metadata descriptor file to
add more information about the data source. This file should have a
header row column named &quot;Column Name&quot; and another column
named &quot;Description&quot;. Each subsequent row should contain
the details for any columns present in the actual data source you
would like to describe.
</li>
</ul>
<div
className={classNames(styles.subtitleButtonContainer, {
[styles.leftAlign]: props.isModal,
})}
>
<DefaultButton
className={styles.linkLikeButton}
onClick={() => setIsDataSourceDetailExpanded(false)}
>
Show less&nbsp;&nbsp;
<Icon iconName="ChevronUp" />
</DefaultButton>
</div>
</>
) : (
<div
className={classNames(styles.subtitleButtonContainer, {
[styles.leftAlign]: props.isModal,
})}
>
<DefaultButton
className={styles.linkLikeButton}
onClick={() => setIsDataSourceDetailExpanded(false)}
onClick={() => setIsDataSourceDetailExpanded(true)}
>
Show less&nbsp;&nbsp;
<Icon iconName="ChevronUp" />
Show more&nbsp;&nbsp;
<Icon iconName="ChevronDown" />
</DefaultButton>
</div>
</>
) : (
<div
className={classNames(styles.subtitleButtonContainer, {
[styles.leftAlign]: props.isModal,
})}
>
<DefaultButton
className={styles.linkLikeButton}
onClick={() => setIsDataSourceDetailExpanded(true)}
>
Show more&nbsp;&nbsp;
<Icon iconName="ChevronDown" />
</DefaultButton>
</div>
)}
</div>
)}
</div>
)}
</div>
);
}
23 changes: 21 additions & 2 deletions packages/core/components/FileDetails/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,15 @@ import FileAnnotationList from "./FileAnnotationList";
import Pagination from "./Pagination";
import useFileDetails from "./useFileDetails";
import { PrimaryButton } from "../Buttons";
import { ModalType } from "../Modal";
import Tooltip from "../Tooltip";
import { ROOT_ELEMENT_ID } from "../../App";
import FileThumbnail from "../../components/FileThumbnail";
import AnnotationName from "../../entity/Annotation/AnnotationName";
import annotationFormatterFactory, { AnnotationType } from "../../entity/AnnotationFormatter";
import useOpenWithMenuItems from "../../hooks/useOpenWithMenuItems";
import { MAX_DOWNLOAD_SIZE_WEB } from "../../services/FileDownloadService";
import { interaction } from "../../state";
import { interaction, provenance } from "../../state";

import styles from "./FileDetails.module.css";

Expand Down Expand Up @@ -127,7 +128,7 @@ export default function FileDetails(props: Props) {
}
}
}
}, [fileDetails, fileDownloadService, isOnWeb, isZarr]);
}, [dispatch, fileDetails, fileDownloadService, isOnWeb, isZarr]);

const processStatuses = useSelector(interaction.selectors.getProcessStatuses);
const openWithMenuItems = useOpenWithMenuItems(fileDetails || undefined);
Expand Down Expand Up @@ -184,6 +185,15 @@ export default function FileDetails(props: Props) {
}, 1000); // 1s, in ms (arbitrary)
}, [dispatch, fileDetails, fileDownloadService.isFileSystemAccessible]);

const onClickProvenance = React.useCallback(async () => {
if (!fileDetails) {
return;
}
// Start generating nodes and edges for selected file
dispatch(provenance.actions.constructProvenanceGraph(fileDetails));
dispatch(interaction.actions.setVisibleModal(ModalType.Provenance));
}, [dispatch, fileDetails]);

return (
<div
className={classNames(styles.root, styles.expandableTransition, props.className)}
Expand Down Expand Up @@ -237,6 +247,15 @@ export default function FileDetails(props: Props) {
menuItems={openWithMenuItems}
/>
</StackItem>
<StackItem>
<PrimaryButton
className={styles.primaryButton}
text="Provenance"
title="Temporary button to display provenance graph"
onClick={onClickProvenance}
// to do: disable if no provenance source
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check with UX, but if the button is still there just disabled (which might not be the best move - unsure) there should be some way to get "Help" for what that button would do

/>
</StackItem>
</Stack>
<p className={styles.fileName}>{fileDetails?.name}</p>
<h4>Information</h4>
Expand Down
5 changes: 5 additions & 0 deletions packages/core/components/Modal/BaseModal/BaseModal.module.css
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@
flex-direction: column;
}

.full-screen {
max-width: 90%;
height: 90%;
}

.scrollable-container {
overflow: hidden;
}
Expand Down
7 changes: 6 additions & 1 deletion packages/core/components/Modal/BaseModal/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { Modal } from "@fluentui/react";
import classNames from "classnames";
import { noop } from "lodash";
import * as React from "react";

Expand All @@ -11,6 +12,7 @@ interface BaseModalProps {
onDismiss?: () => void;
title?: string;
isStatic?: boolean; // Not draggable
isFullScreen?: boolean; // Override width constraints
}

/**
Expand All @@ -25,7 +27,9 @@ export default function BaseModal(props: BaseModalProps) {
<Modal
isOpen
onDismiss={onDismiss}
containerClassName={styles.container}
containerClassName={classNames(styles.container, {
[styles.fullScreen]: props.isFullScreen,
})}
titleAriaId={titleId}
overlay={{ className: styles.overlay }}
>
Expand All @@ -47,4 +51,5 @@ BaseModal.defaultProps = {
footer: null,
onDismiss: noop,
isStatic: false,
isFullScreen: false,
};
Loading