Skip to content

Use the new data loaders in the refactored data explorer. #3492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 67 commits into
base: refactor24-data-explorer
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
f24a033
Add a dummy new data explorer.
jamescrake-merani Jan 24, 2025
b4ffb03
Hook up an event for new perspective.
jamescrake-merani Feb 7, 2025
76ab33b
File chooser for load data.
jamescrake-merani Jun 19, 2025
26a9311
Load based on extension.
jamescrake-merani Jun 19, 2025
2dcc17c
Add all data files to the manager.
jamescrake-merani Jun 19, 2025
2881004
End early on error.
jamescrake-merani Jun 19, 2025
f32eb5b
Fixed some issues from the rebase.
jamescrake-merani Jun 20, 2025
b796902
Missing method from the rebase.
jamescrake-merani Jun 20, 2025
7a9292e
I have no idea why this line was here.
jamescrake-merani Jun 20, 2025
d91c588
Something weird has happened here.
jamescrake-merani Jun 20, 2025
edeff08
Fixed weird code getting removed.
jamescrake-merani Jun 20, 2025
abbc52f
Support loading in ASCII files.
jamescrake-merani Jun 20, 2025
a5e19f8
Full stop not needed here.
jamescrake-merani Jun 20, 2025
4bac60d
Remove old imports.
jamescrake-merani Jun 25, 2025
54ab3fe
Load ascii with default params.
jamescrake-merani Jun 25, 2025
f14a042
Pick out the sasdata object.
jamescrake-merani Jun 25, 2025
1a9e1ec
Use the basename for this dict.
jamescrake-merani Jun 25, 2025
fbfd28e
Initial data viewer dialog.
jamescrake-merani Jun 25, 2025
4d16fed
Add filename label and layout.
jamescrake-merani Jun 25, 2025
d747ced
Rename filenameLabel to nameLabel.
jamescrake-merani Jun 25, 2025
cc5a1f8
Ruff format.
jamescrake-merani Jun 25, 2025
3493f46
Removed unusued import.
jamescrake-merani Jun 25, 2025
4c7350a
Ruff format.
jamescrake-merani Jun 25, 2025
114cb64
Added the view data action.
jamescrake-merani Jun 25, 2025
792bc98
View data from the context menu.
jamescrake-merani Jun 25, 2025
5de6199
Add view data action to menu.
jamescrake-merani Jun 25, 2025
ecbd9f8
Hook up event.
jamescrake-merani Jun 25, 2025
07f16dd
Fixed typo.
jamescrake-merani Jun 25, 2025
57cbd2a
Set parent of layout properly.
jamescrake-merani Jun 25, 2025
9590fd2
Use name for name label.
jamescrake-merani Jun 25, 2025
092ceca
Added an advance load button.
jamescrake-merani Jul 2, 2025
f282268
Hook up the advanced load menu item.
jamescrake-merani Jul 2, 2025
4528a9a
Load in data from the dialog.
jamescrake-merani Jul 2, 2025
1da4e4e
Code for the widgets in the sketch.
jamescrake-merani Jul 3, 2025
eb84dda
Add these to the layout.
jamescrake-merani Jul 3, 2025
60b0ef4
Just use the name of the dataset type.
jamescrake-merani Jul 3, 2025
650324e
The table should span 2 rows.
jamescrake-merani Jul 3, 2025
141c0f2
Added close button..
jamescrake-merani Jul 3, 2025
727698f
Implement build table function.
jamescrake-merani Jul 4, 2025
608c9cc
Just use the value.
jamescrake-merani Jul 4, 2025
048e6b8
Disable SESANS for the ascii dialog.
jamescrake-merani Jul 4, 2025
d9454e5
Set column, and row count.
jamescrake-merani Jul 4, 2025
7663fec
Can't subscript a dict, so do this instead.
jamescrake-merani Jul 4, 2025
41e9837
Make the table readonly.
jamescrake-merani Jul 4, 2025
daf562d
Wrong field.
jamescrake-merani Jul 4, 2025
e5ab531
Has to be converted into str.
jamescrake-merani Jul 4, 2025
2d78abb
Try to make the table take the full width.
jamescrake-merani Jul 4, 2025
d6ef6a1
This is the right way to resize properly.
jamescrake-merani Jul 4, 2025
f3fb6f8
Hook up the open metadata explorer btn.
jamescrake-merani Jul 4, 2025
e0e8a8b
Ruff format.
jamescrake-merani Jul 8, 2025
76a45fe
Another Ruff format :P
jamescrake-merani Jul 8, 2025
dfddec8
Replaced the definition of metadata as dict.
jamescrake-merani Jul 9, 2025
77bac2f
Fix recursion.
jamescrake-merani Jul 9, 2025
3e1dc07
Whoops wrong variable.
jamescrake-merani Jul 9, 2025
f880676
Fixes.
jamescrake-merani Jul 9, 2025
40993b3
Finally found a solution that works :P
jamescrake-merani Jul 9, 2025
1701713
Quantites should be considered leaves as well.
jamescrake-merani Jul 10, 2025
15a767e
Added possible hdf extension for hdf5 files.
jamescrake-merani Jul 10, 2025
4ea4c1d
Return if we didn't select a file.
jamescrake-merani Jul 10, 2025
61e715a
Support loading multiple files at once.
jamescrake-merani Jul 10, 2025
8c6c813
ASCII files can use the CSV format.
jamescrake-merani Jul 10, 2025
484c63f
Code review suggestions.
jamescrake-merani Jul 21, 2025
0e7f27b
Get rid of unused import.
jamescrake-merani Jul 21, 2025
4055b2f
Set title of data viewer window.
jamescrake-merani Jul 21, 2025
8fad058
Remove the old if statement.
jamescrake-merani Jul 21, 2025
fe122f4
Loop is redundant.
jamescrake-merani Jul 21, 2025
b5b598b
Just mutate dataset types.
jamescrake-merani Jul 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions src/ascii_dialog/dialog.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from metadata_filename_gui.metadata_tree_data import initial_metadata_dict
from sasdata.ascii_reader_metadata import AsciiReaderMetadata
from ascii_dialog.constants import TABLE_MAX_ROWS, NOFILE_TEXT
from contextlib import suppress
import re

dataset_dictionary = dict([(dataset.name, dataset) for dataset in [one_dim, two_dim, sesans]])
Expand Down Expand Up @@ -64,8 +65,10 @@ def __init__(self):
self.dataset_layout = QHBoxLayout()
self.dataset_label = QLabel("Dataset Type")
self.dataset_combobox = QComboBox()
for name in dataset_types:
self.dataset_combobox.addItem(name)
# TODO: Temporarily exclude SESANS until that's been fixed.
with suppress(ValueError):
dataset_types.remove('SESANS')
self.dataset_combobox.addItems(dataset_types)
self.dataset_layout.addWidget(self.dataset_label)
self.dataset_layout.addWidget(self.dataset_combobox)

Expand Down
14 changes: 10 additions & 4 deletions src/sas/data_explorer_menu.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,27 +6,33 @@
from src.sas.refactored import Perspective
from dataclasses import dataclass


# TODO: Could we perhaps enforce that, if the action is send_to, the action_data
# should not be None?
@dataclass
class DataExplorerMenuAction():
action: Literal['remove', 'send_to']
class DataExplorerMenuAction:
action: Literal["remove", "send_to", "view_data"]
action_data: Perspective | None = None


class DataExplorerMenu(QMenu):
def __init__(self, parent: QWidget, data_manager: NewDataManager, send_to: bool):
super().__init__(parent)

remove_data = QAction("Remove", parent)
# remove_data.setData('remove')
remove_data.setData(DataExplorerMenuAction('remove'))
remove_data.setData(DataExplorerMenuAction("remove"))

view_data = QAction("View Data", parent)
view_data.setData(DataExplorerMenuAction("view_data"))
# TODO: There will be loads more

if send_to:
send_to_menu = self.addMenu("Send To")

for perspective in data_manager.all_perspectives:
new_action = QAction(perspective.formatName, parent)
new_action.setData(DataExplorerMenuAction('send_to', perspective))
new_action.setData(DataExplorerMenuAction("send_to", perspective))
send_to_menu.addAction(new_action)
self.addAction(remove_data)
self.addAction(view_data)
43 changes: 35 additions & 8 deletions src/sas/data_explorer_tree.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,19 @@
from typing_extensions import cast
from PySide6.QtCore import Qt, Signal
from PySide6.QtGui import QCursor
from PySide6.QtWidgets import QAbstractItemView, QMessageBox, QTreeWidget, QTreeWidgetItem, QWidget
from PySide6.QtWidgets import (
QAbstractItemView,
QTreeWidget,
QTreeWidgetItem,
QWidget,
)
from sasdata.data import SasData
from sas.data_manager import NewDataManager as DataManager, TrackedData
from sas.refactored import Perspective
from src.sas.data_explorer_error_message import DataExplorerErrorMessage
from src.sas.data_explorer_menu import DataExplorerMenu, DataExplorerMenuAction
from sas.qtgui.MainWindow.DataViewer import DataViewer


# TODO: Is this the right place for this?
def tracked_data_name(data: TrackedData) -> str:
Expand All @@ -15,14 +22,19 @@ def tracked_data_name(data: TrackedData) -> str:
else:
return data.formatName


class DataExplorerTree(QTreeWidget):
current_datum_removed = Signal()
view_data_activated = Signal()

def __init__(self, data_manager: DataManager, parent: QWidget | None = None) -> None:
def __init__(
self, data_manager: DataManager, parent: QWidget | None = None
) -> None:
super().__init__(parent)
self._data_manager = data_manager
_ = self._data_manager.new_data.connect(self.addToTable)
_ = self._data_manager.data_removed.connect(self.removeFromTable)
_ = self.view_data_activated.connect(self.showViewData)

self.setContextMenuPolicy(Qt.ContextMenuPolicy.CustomContextMenu)
self.setSelectionMode(QAbstractItemView.SelectionMode.ExtendedSelection)
Expand Down Expand Up @@ -55,13 +67,17 @@ def removeAssociation(self, datum1: TrackedData, datum2: TrackedData):
# TODO: Again, order.
self.removeFromTable(datum2, root_datum=datum1)


def addToTable(self, datum: TrackedData):
item = QTreeWidgetItem([tracked_data_name(datum)])
item.setData(0, Qt.ItemDataRole.UserRole, datum)
self.addTopLevelItem(item)

def removeFromTable(self, datum: TrackedData, starting_root: QTreeWidgetItem | None = None, root_datum: TrackedData | None = None):
def removeFromTable(
self,
datum: TrackedData,
starting_root: QTreeWidgetItem | None = None,
root_datum: TrackedData | None = None,
):
"""The root_datum param is needed when you want to delete something from the tree that has a certain root item.
This is mostly useful for getting rid of associations."""
root = self.invisibleRootItem() if starting_root is None else starting_root
Expand All @@ -71,7 +87,10 @@ def removeFromTable(self, datum: TrackedData, starting_root: QTreeWidgetItem | N
for i in range(root.childCount()):
item = root.child(i)
item_datum = cast(TrackedData, item.data(0, Qt.ItemDataRole.UserRole))
if item_datum == datum and (not root_datum is None or root_datum == root.data(0, Qt.ItemDataRole.UserRole)):
if item_datum == datum and (
root_datum is not None
or root_datum == root.data(0, Qt.ItemDataRole.UserRole)
):
# Need to defer this to later so we don't delete data while we're doing a for loop over it.
to_remove.append(item)
elif item.childCount() != 0:
Expand All @@ -89,10 +108,12 @@ def showContextMenu(self):
result: DataExplorerMenuAction = action.data()
errors: list[ValueError] = []
match result.action:
case 'remove':
case "remove":
# TODO: Work for all data.
self.current_datum_removed.emit()
case 'send_to':
case "view_data":
self.view_data_activated.emit()
case "send_to":
# TODO: This cast might not be necessary.
to_perspective = cast(Perspective, result.action_data)
# TODO: I'm a bit worried about potential repetition if there are more actions here. Will need to
Expand All @@ -106,6 +127,10 @@ def showContextMenu(self):
box = DataExplorerErrorMessage(self, errors)
box.show()

def showViewData(self):
viewer = DataViewer(self.currentTrackedDatum)
viewer.exec()

@property
def currentTrackedDatum(self) -> TrackedData | None:
if len(self.currentTrackedData) == 0:
Expand All @@ -121,7 +146,9 @@ def currentTrackedData(self) -> list[TrackedData]:

# Annoyingly, there is no way to get all the items in a tree. So we have to
# do this recursively instead.
def setCurrentTrackedDatum(self, datum: TrackedData, root: QTreeWidgetItem | None = None):
def setCurrentTrackedDatum(
self, datum: TrackedData, root: QTreeWidgetItem | None = None
):
if root is None:
root = self.invisibleRootItem()
for i in range(root.childCount()):
Expand Down
58 changes: 58 additions & 0 deletions src/sas/qtgui/MainWindow/DataViewer.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
from PySide6.QtWidgets import (
QDialog,
QLabel,
QGridLayout,
QPushButton,
QTableWidget,
QTableWidgetItem,
QHeaderView,
)
from sasdata.data import SasData
from sas.qtgui.MainWindow.MetadataExplorer import MetadataExplorer


class DataViewer(QDialog):
def __init__(self, to_view: SasData):
super().__init__()
self.to_view = to_view
self.layout = QGridLayout(self)

self.nameLabel = QLabel(f"Name: {self.to_view.name}")
self.viewMetadataButton = QPushButton("View Metadata")
self.viewMetadataButton.clicked.connect(self.openMetadataExplorer)
self.dataTypeLabel = QLabel(
f"Type: {self.to_view.dataset_type.name}"
) # TODO: Probably a better way of printing this
self.dataTable = QTableWidget()
self.buildTable()
self.closeButton = QPushButton("Close")
self.closeButton.clicked.connect(self.close)

self.setWindowTitle("Data Viewer")
self.layout.addWidget(self.nameLabel, 0, 0, 1, 1)
self.layout.addWidget(self.viewMetadataButton, 0, 1, 1, 1)
self.layout.addWidget(self.dataTypeLabel, 1, 0, 1, 1)
self.layout.addWidget(self.dataTable, 2, 0, 1, 2)
self.layout.addWidget(self.closeButton, 3, 0, 1, 2)

def buildTable(self):
# Make the table readonly
self.dataTable.setEditTriggers(QTableWidget.EditTrigger.NoEditTriggers)
columns = self.to_view._data_contents.keys()
self.dataTable.setColumnCount(len(columns))
# NOTE: Assumes each column has the same amount of rows, which should be
# the case, although perhaps we should validate this.
self.dataTable.setRowCount(
len(next(iter(self.to_view._data_contents.values())).value)
)
self.dataTable.setHorizontalHeaderLabels(columns)
self.dataTable.horizontalHeader().setSectionResizeMode(
QHeaderView.ResizeMode.Stretch
)
for i, data in enumerate(self.to_view._data_contents.values()):
for j, datum in enumerate(data.value):
self.dataTable.setItem(j, i, QTableWidgetItem(str(datum)))

def openMetadataExplorer(self):
explorer = MetadataExplorer(self.to_view.metadata, self.to_view.name)
explorer.exec()
5 changes: 5 additions & 0 deletions src/sas/qtgui/MainWindow/GuiManager.py
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ def addWidgets(self):
# self.filesWidget = DataExplorerWindow(self._parent, self, manager=self._data_manager)
# TODO: Is this a good opportunity to change this name? dataExplorer would be better I think.
self.filesWidget = NewDataExplorer(self._data_manager ,self._parent)
self.filesWidget.new_perspective.connect(self.new_perspective)
ObjectLibrary.addObject('DataExplorer', self.filesWidget)

self.dockedFilesWidget = QDockWidget("Data Explorer", self._workspace)
Expand Down Expand Up @@ -778,6 +779,7 @@ def addTriggers(self):

# File
self._workspace.actionLoadData.triggered.connect(self.actionLoadData)
self._workspace.actionAdvanced_Load.triggered.connect(self.actionAdvancedLoad)
self._workspace.actionLoad_Data_Folder.triggered.connect(self.actionLoad_Data_Folder)
self._workspace.actionOpen_Project.triggered.connect(self.actionOpen_Project)
self._workspace.actionOpen_Analysis.triggered.connect(self.actionOpen_Analysis)
Expand Down Expand Up @@ -869,6 +871,9 @@ def actionLoadData(self):
"""
self.filesWidget.loadFile()

def actionAdvancedLoad(self):
self.filesWidget.onAdvancedLoad()

def actionLoad_Data_Folder(self):
"""
Menu File/Load Data Folder
Expand Down
49 changes: 21 additions & 28 deletions src/sas/qtgui/MainWindow/MetadataExplorer.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,37 +10,27 @@
QVBoxLayout,
)
from sasdata.metadata import MetaNode, Metadata
from sasdata.quantities.quantity import Quantity
from sasdata.temp_xml_reader import load_data


def convert_raw_to_dict(to_convert: MetaNode) -> dict:
# converted = {to_convert.name: to_convert.contents}
if isinstance(to_convert.contents, str) or isinstance(
to_convert.contents, Quantity
):
return {to_convert.name: to_convert.contents}
value = {}
# We can now assume that every content is a MetaNode as per the typing.
for content in to_convert.contents:
value = value | convert_raw_to_dict(content)
return {to_convert.name: value}


def metadata_as_dict(to_convert: object):
"""This is a custom implementation of asdict from dataclasses. The key
difference is that MetaNode class objects are preserved (as well as other
leaf nodes), but everything else is converted into a dict. This makes it
easier to iterate over.

"""
if isinstance(to_convert, list) and all([hasattr(item, '__dict__') for item in to_convert]):
converted_dicts: list[dict[str, object]] = []
for item in to_convert:
converted_dicts.append(item.__dict__)
elif hasattr(to_convert, "__dict__"):
converted_dicts = [to_convert.__dict__]
else:
return to_convert
for single_dict in converted_dicts:
for key, value in single_dict.items():
# This if statement looks for a meta node that is a child node, and leaves it as is (i.e. it doesn't turn it
# into a dict). Some meta nodes contain other meta nodes, so we need to add a condition for this.
if not (
value is dict
or (
isinstance(value, MetaNode)
and [isinstance(content, MetaNode) for content in value.contents]
)
):
single_dict[key] = metadata_as_dict(value)
return converted_dicts[0] if len(converted_dicts) == 0 else converted_dicts
converted = to_convert.__dict__
converted["raw"] = convert_raw_to_dict(converted["raw"])
return converted


class MetadataExplorer(QDialog):
Expand Down Expand Up @@ -82,7 +72,10 @@ def buildTree(
dicts = [current_item]
for single_dict in dicts:
for key, value in single_dict.items():
if isinstance(value, dict) or (isinstance(value, list) and any([isinstance(member, dict) for member in value])):
if isinstance(value, dict) or (
isinstance(value, list)
and any([isinstance(member, dict) for member in value])
):
dict_root = QTreeWidgetItem([key])
table_root.addChild(dict_root)
self.buildTree(dict_root, value)
Expand Down
8 changes: 7 additions & 1 deletion src/sas/qtgui/MainWindow/UI/MainWindowUI.ui
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,15 @@
<x>0</x>
<y>0</y>
<width>915</width>
<height>20</height>
<height>22</height>
</rect>
</property>
<widget class="QMenu" name="menu_File">
<property name="title">
<string>&amp;File</string>
</property>
<addaction name="actionLoadData"/>
<addaction name="actionAdvanced_Load"/>
<addaction name="actionLoad_Data_Folder"/>
<addaction name="separator"/>
<addaction name="actionOpen_Project"/>
Expand Down Expand Up @@ -650,6 +651,11 @@
<string>Dev Tools</string>
</property>
</action>
<action name="actionAdvanced_Load">
<property name="text">
<string>Advanced Load</string>
</property>
</action>
</widget>
<resources/>
<connections/>
Expand Down
Loading
Loading