Node -> TypeVar("T", bound=Node) in tree.construct? #247
Conversation
|
Thanks for spotting this! A few amendments to the commit
I will review more when I get to my laptop, but on your end can you run Edit: You mentioned about For the enhanced |
|
Amazing responsiveness!
Back to from typing import TypeVar, Union, List
import re
from os import walk as os_walk
from os.path import relpath as os_path_relpath, join as os_path_join
from pathlib import Path
from tempfile import mkdtemp
from subprocess import run as subprocess_run
from shutil import rmtree
from bigtree.node.basenode import BaseNode
from bigtree.node.node import Node
from bigtree.tree.construct import list_to_tree as bigtree_list_to_tree
from mytree.node.weightednode import WeightedNode
__all__ = [
"dir_to_tree",
"list_to_tree",
]
T = TypeVar("T", bound=WeightedNode)
PathType = Union[str, Path]
def _read_tree_dir(root_path: PathType) -> List[str]:
"""root_path/root_name/... (root_path excludes the root)"""
root_path = Path(root_path)
output_dirs = []
for root, dirs, files in os_walk(root_path):
for dir in dirs:
full_path = os_path_relpath(os_path_join(root, dir), root_path)
output_dirs.append(full_path)
return output_dirs
def dir_to_tree(root_path: PathType) -> T:
"""root_path/root_name/... (root_path excludes the root)"""
return bigtree_list_to_tree(
_read_tree_dir(root_path),
sep="/",
duplicate_name_allowed=True,
node_type=WeightedNode,
)
def _expand_braces(input_string: str) -> List[str]:
# Create a temporary directory
temp_dir = mkdtemp()
try:
# Construct the mkdir -p command
command = f"mkdir -p {os_path_join(temp_dir, input_string)}"
# Run the command
subprocess_run(command, shell=True, check=True)
# Walk through the created directories and collect them
return _read_tree_dir(temp_dir)
finally:
# Clean up the temporary directory
rmtree(temp_dir)
def list_to_tree(
paths: List[str],
sep: str = "/",
duplicate_name_allowed: bool = True,
expand: bool = True,
) -> T:
if expand:
paths = sum([_expand_braces(path) for path in paths], [])
return bigtree_list_to_tree(
paths, sep=sep, duplicate_name_allowed=True, node_type=WeightedNode
)This way, the example of the doc could be done in one string with: root = list_to_tree(["a/{b/{d,e/{g,h}},c/f}"])My trees are quite big with lots of duplicate names, so it's very useful. |
|
Thanks for making the amendments so quickly 👍 . I just got on my laptop and running Attaching the mypy errors below. Actually for TypeVar, the |
|
Ok, I'm very sorry for the mess. I'm going to follow the guidelines now and come back to you then. |
|
No worries, I'll also look into this. If you don't mind, can send how you are using |
|
I implemented a fix in v0.18.3, do upgrade the package with |
|
@DesmetQuentin Hi following up on the However I see that your code has a few extra stuff such as WeightedNode and also the interpreting of curly braces notation - let me know if those use cases are worth diving into! |
|
@kayjan I thank you for questioning this. I've looked a bit more into it and now give you a "report". Topic: Just to be sure of what we are talking about, I'm talking about construct functions here, not export. I'm interested into using the unix-style brace expansion for constructing tree faster, in cases like mines where I have many duplicate names in big trees. It's essentially about an enhanced Previouly: To do this, I initially lazily came up with applying the Now: I actually found this from typing import TypeVar, List, Type
from braceexpand import braceexpand
from bigtree import Node, list_to_tree as bigtree_list_to_tree
T = TypeVar("T", bound=Node)
def list_to_tree(
paths: List[str],
sep: str = "/",
duplicate_name_allowed: bool = True,
expand: bool = True,
node_type: Type[T] = Node, # type: ignore[assignment]
) -> T:
if expand:
paths = sum([list(braceexpand(path)) for path in paths], [])
return bigtree_list_to_tree(
paths, sep=sep, duplicate_name_allowed=duplicate_name_allowed, node_type=node_type
)Application: In practice, I prefer to construct my tree from a file storing a unix-brace-based expression, but with break lines and comments. Very concretely here is my file: Final/{
{
"Temporal variability"/ # here it is a nice comment
{PAC,AND,SUN,SCS,SCB,IND,JAV,SAH},
"Spatial variability"/
{DJF,JJA}
}/
{CC,NSD} # hey, modify here!
,
'Mean value'/{
"Temporal aspect"/
{PAC,AND,SUN,SCS,SCB,IND,JAV,SAH},
"Spatial aspect"/
{DJF,JJA}
}/
MB
}After parsing the file: After applying And here is my Then? After this, I apply my weights and stuff for my own study case (still in development), but you got the idea. I don't know if you want to include this inside |
|
The Is this braces notation a formal type of tree representation? There is another form of tree representation with Newick which is similar but uses a different type of bracketing, and also not as "automated" as yours in the sense where |
|
I don't know if it was thought for making trees, but it can be used for it. In a unix terminal, |
2 participants
Hi there,
Well first of all thank you for this
bigtreepackage, it's been helping me a lot for my own application so far. I am currently working on a subclass of Node aiming to implement weighted trees for aggregating data using weighted averages from the leaves to the roots. I am quite grateful that you have usedTypeVar("T", bound=Node)most of the time. It made my coding much simpler and saved me a lot of time. However, in my current implementation,mypycomplains because even though I set upnode_type = WeightedNode(my child class ofNode), the output of, e.g.list_to_treeis always aNode. Is there a reason whyconstruct.pydid not follow this convention of using aTypeVar("T", bound=Node)? If not, then I propose to apply the convention to this file too, through the present pull request.Otherwise, I have some other construct function propositions if you are interested, such as a
dir_to_treebuilding a tree from an existing folder architecture, or an enhanced version oflist_to_treewhich employs the same convention as the bash commandmkdirwith the braces. We can discuss about it further if you are interested.Regards,
Q. Desmet