Skip to content

Mega rewrite #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 98 commits into from
May 5, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
9f15783
initial commit of rewrite
pufferfish101007 Dec 27, 2024
4737e34
commit number 2: full testing support for validity of wasm instructions
pufferfish101007 Dec 28, 2024
f8df9bc
commit 3: auto-generate opcode enum & related functions
pufferfish101007 Dec 28, 2024
7946cad
basic looks_say, js type checking, and Other Things
pufferfish101007 Jan 1, 2025
0146d53
start implementing IR
pufferfish101007 Jan 2, 2025
5462a0c
format and add remaining primitive number blocks
pufferfish101007 Jan 3, 2025
75fe2a1
getting closer
pufferfish101007 Jan 3, 2025
02290b0
try to avoid circular strong references
pufferfish101007 Jan 4, 2025
b2671ae
mostly finish putting everything together
pufferfish101007 Jan 4, 2025
c7f0729
make things work(ish) in the playground
pufferfish101007 Jan 4, 2025
9ae58b1
wasm almost functioning correctly
pufferfish101007 Jan 5, 2025
6dad059
ensure finished threads exit successfully
pufferfish101007 Jan 5, 2025
ec72b12
fix things (incl. use i32 rather than i64)
pufferfish101007 Jan 8, 2025
c78d090
start implementing strings; expose settings/flags to js; generalise r…
pufferfish101007 Jan 12, 2025
9c3f55d
start implementing casts
pufferfish101007 Jan 19, 2025
3b22e82
(hopefully) fix casts
pufferfish101007 Jan 25, 2025
7c2e1c6
update casting system to work properly
pufferfish101007 Feb 17, 2025
f61bdb6
add instructions on adding new blocks
pufferfish101007 Feb 17, 2025
d140faf
CI overhaul
pufferfish101007 Feb 18, 2025
753e2a2
auto-track implemented blocks
pufferfish101007 Feb 18, 2025
c0ee513
don't specify an absolute href for example links
pufferfish101007 Feb 18, 2025
52b88ee
use hash history
pufferfish101007 Feb 18, 2025
51b56c1
fix some bits and bobs
pufferfish101007 Feb 18, 2025
aaceeb6
convert nans to 0 (resolves hyperquark/hyperquark#17)
pufferfish101007 Feb 19, 2025
f553727
remove unused files
pufferfish101007 Feb 19, 2025
d0b0011
tighten bounds on output type of operator_add
pufferfish101007 Feb 19, 2025
e58af33
update internal documentation and internal imports + README
pufferfish101007 Feb 19, 2025
f441e7e
operator_divide
pufferfish101007 Feb 19, 2025
795beab
consider that addition can be nan for inf+(-inf)
pufferfish101007 Feb 19, 2025
b95add0
operator_subtract
pufferfish101007 Feb 19, 2025
e851117
bad input type is a bug not a todo
pufferfish101007 Feb 19, 2025
5abf1de
operator_multiply and sensing_dayssince2000
pufferfish101007 Feb 19, 2025
f0ec59e
begin implementing variables
pufferfish101007 Feb 21, 2025
e00e465
allow strings in variables
pufferfish101007 Feb 21, 2025
fcaa82b
implement operator_lt
pufferfish101007 Feb 22, 2025
79096e7
implement control_if
pufferfish101007 Feb 22, 2025
88340d0
remove unused file
pufferfish101007 Feb 22, 2025
4f686a0
allow for empty substacks or (predicate) inputs
pufferfish101007 Feb 22, 2025
7f0afc6
update CI
pufferfish101007 Feb 22, 2025
3632137
lint
pufferfish101007 Feb 22, 2025
9183571
`enum-field-getter`: fix typo in `repository` field
pufferfish101007 Feb 22, 2025
30577b4
implement operator_not
pufferfish101007 Feb 22, 2025
43b854d
control_repeat
pufferfish101007 Feb 22, 2025
2a3378c
remove unnecessary logs
pufferfish101007 Feb 22, 2025
9794eec
don't track generated files
pufferfish101007 Feb 23, 2025
a43e43b
ci: run tests
pufferfish101007 Feb 23, 2025
ee6908b
update operator_join test
pufferfish101007 Feb 23, 2025
32c686d
implement string -> int cast
pufferfish101007 Feb 24, 2025
c26118a
ensure that int->int cast works
pufferfish101007 Feb 24, 2025
ef99a11
ignore doctest
pufferfish101007 Feb 24, 2025
758671f
tests: update input tyoe filtering order
pufferfish101007 Feb 24, 2025
6adaff7
properly initialise loop counter variables
pufferfish101007 Feb 25, 2025
c352ac4
fix wrapped casting
pufferfish101007 Feb 24, 2025
5733b19
don't error on top-level special block
pufferfish101007 Feb 26, 2025
962a457
sort of variable type specialisation
pufferfish101007 Mar 1, 2025
b7dbc9c
update ci
pufferfish101007 Mar 1, 2025
0222ae5
use typed function references for threads
pufferfish101007 Mar 16, 2025
e6033a1
remove unneeded test
pufferfish101007 Mar 17, 2025
85d94f5
add operator_gt
pufferfish101007 Mar 23, 2025
72a4417
fix operator_gt
pufferfish101007 Mar 23, 2025
06cd8b0
implement yielding
pufferfish101007 Apr 9, 2025
13aaeeb
handle project id input properly
pufferfish101007 Apr 10, 2025
7be5480
lint
pufferfish101007 Apr 10, 2025
ab6eaf4
mostly implement warped procedures
pufferfish101007 Apr 12, 2025
d1299c6
run control_repeat correct amount of times
pufferfish101007 Apr 13, 2025
d6b80dc
fix logic for determining whether to produce wasm func for step
pufferfish101007 Apr 13, 2025
3aedbec
allow for empty argumentids/argumentnames arrays
pufferfish101007 Apr 13, 2025
3d3a572
support global variables
pufferfish101007 Apr 13, 2025
7463de9
fix stop function
pufferfish101007 Apr 14, 2025
423f517
fix some nested loops
pufferfish101007 Apr 14, 2025
8063c13
fix scheduler
pufferfish101007 Apr 14, 2025
1a3adfb
use locals for loops
pufferfish101007 Apr 14, 2025
f59dcdc
use bubbles; start playing around with binaryen
pufferfish101007 Apr 18, 2025
f21b518
update ignored tests
pufferfish101007 Apr 18, 2025
07df55e
add looks_think
pufferfish101007 Apr 18, 2025
ee440c8
fix loops (again)
pufferfish101007 Apr 19, 2025
6889c2f
implement control_if_else
pufferfish101007 Apr 19, 2025
ef12803
implement data_changevariableby
pufferfish101007 Apr 20, 2025
cdeb04a
implement operator_and & operator_or
pufferfish101007 Apr 20, 2025
53697e6
use js string builtins for concat
pufferfish101007 Apr 20, 2025
195f554
implement more string operations
pufferfish101007 Apr 20, 2025
c5af91f
more strings
pufferfish101007 Apr 20, 2025
8fddc73
update block list colours for procedures
pufferfish101007 Apr 20, 2025
7ee5ac7
skip failing test
pufferfish101007 Apr 20, 2025
42f900a
optimise wasm module with binaryen
pufferfish101007 Apr 20, 2025
e97a83f
implement operator_equals
pufferfish101007 Apr 21, 2025
4de2f7d
implement control_repeat_until
pufferfish101007 Apr 21, 2025
8f425e7
condense loop generation into one function
pufferfish101007 Apr 21, 2025
3f134d4
improve binaryen opts; make wasm-opt optional
pufferfish101007 Apr 21, 2025
b6ae18b
fix warped control_repeat_until
pufferfish101007 Apr 21, 2025
55c712b
don't use colons in directory names
pufferfish101007 Apr 23, 2025
a32d96f
move more flag logic to rust
pufferfish101007 Apr 23, 2025
7e8a61d
run wasm-opt in deploy with latest binaryen
pufferfish101007 Apr 24, 2025
43a1949
actually use call_ref when using typed_function references
pufferfish101007 Apr 28, 2025
0b6587f
add option to use old call_indirect scheduler, and fix binaryen better
pufferfish101007 May 2, 2025
03000d1
fix binaryen dependency name
pufferfish101007 May 3, 2025
0b32a4a
add rust-toolchain.toml
pufferfish101007 May 3, 2025
c52ee5d
mega lint
pufferfish101007 May 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
mostly finish putting everything together
  • Loading branch information
pufferfish101007 committed May 5, 2025
commit b2671aead1f564a2c19ca254df20936a8ff9adc4
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ Compile scratch projects to WASM
- [Rust](https://rust-lang.org) (v1.65.0 or later)
- the `wasm32-unknown-unknown` target (`rustup target add wasm32-unknown-unknown`)
- wasm-bindgen-cli (`cargo install -f wasm-bindgen-cli`)
- ezno (`cargo install ezno`)
- wasm-opt (install binaryen using whatever package manager you use)

## Building
Expand Down
2 changes: 1 addition & 1 deletion src/instructions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@

use crate::prelude::*;

mod hq;
mod looks;
mod math;
mod operator;
#[macro_use]
mod utilities;
Expand Down
2 changes: 2 additions & 0 deletions src/instructions/hq.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
pub mod float;
pub mod integer;
File renamed without changes.
File renamed without changes.
4 changes: 0 additions & 4 deletions src/instructions/math.rs

This file was deleted.

33 changes: 0 additions & 33 deletions src/instructions/math/positive_number.rs

This file was deleted.

30 changes: 0 additions & 30 deletions src/instructions/math/whole_number.rs

This file was deleted.

34 changes: 14 additions & 20 deletions src/instructions/utilities.rs
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ macro_rules! instructions_test {
let type_registry = Rc::new(TypeRegistry::new());
let external_functions = Rc::new(ExternalFunctionMap::new());
let types: &[IrType] = &[$($type_arg,)*];
let step_func = StepFunc::new_with_param_count(types.len(), type_registry.clone(), external_functions.clone()).unwrap();
let step_func = StepFunc::new(type_registry.clone(), external_functions.clone());
let wasm_result = wasm(&step_func, Rc::new([$($type_arg,)*]), $(&$fields)?);
match (output_type_result.clone(), wasm_result.clone()) {
(Err(..), Ok(..)) | (Ok(..), Err(..)) => panic!("output_type result doesn't match wasm result for type(s) {:?}:\noutput_type: {:?},\nwasm: {:?}", ($($type_arg,)*), output_type_result, wasm_result),
Expand Down Expand Up @@ -113,8 +113,14 @@ macro_rules! instructions_test {
};
let type_registry = Rc::new(TypeRegistry::new());
let external_functions = Rc::new(ExternalFunctionMap::new());
let wasm_proj = $crate::wasm::WasmProject::new(Default::default(), $crate::wasm::ExternalEnvironment::WebBrowser);
let types: &[IrType] = &[$($type_arg,)*];
let step_func = StepFunc::new_with_param_count(types.len(), type_registry.clone(), external_functions.clone())?;
let params = [$($type_arg,)*].into_iter().map(|ty| wasm_proj.ir_type_to_wasm(ty)).collect::<HQResult<Vec<_>>>()?;
let result = match output_type {
Some(output) => Some(wasm_proj.ir_type_to_wasm(output)?),
None => None,
};
let step_func = StepFunc::new_with_types(params.into(), result, Rc::clone(&type_registry), external_functions.clone())?;
let wasm = match wasm(&step_func, Rc::new([$($type_arg,)*]), $(&$fields)?) {
Ok(a) => a,
Err(_) => {
Expand All @@ -126,34 +132,22 @@ macro_rules! instructions_test {
step_func.add_instructions([Instruction::LocalGet(i.try_into().unwrap())])
}
step_func.add_instructions(wasm);
let func = step_func.finish();

let wasm_proj = $crate::wasm::WasmProject::new(Default::default(), $crate::wasm::ExternalEnvironment::WebBrowser);

let mut module = Module::new();

let mut imports = ImportSection::new();
let mut types = TypeSection::new();
let mut functions = FunctionSection::new();
let mut codes = CodeSection::new();

Rc::unwrap_or_clone(external_functions).finish(&mut imports, type_registry.clone())?;

let mut types = TypeSection::new();
let params = [$($type_arg,)*].into_iter().map(|ty| wasm_proj.ir_type_to_wasm(ty)).collect::<HQResult<Vec<_>>>()?;
let results = match output_type {
Some(output) => vec![wasm_proj.ir_type_to_wasm(output)?],
None => vec![],
};
let step_type_index = type_registry.type_index(params, results)?;
Rc::unwrap_or_clone(external_functions).finish(&mut imports, type_registry.clone())?;
step_func.finish(&mut functions, &mut codes)?;
Rc::unwrap_or_clone(type_registry).finish(&mut types);
module.section(&types);

module.section(&types);
module.section(&imports);

let mut functions = FunctionSection::new();
functions.function(step_type_index);
module.section(&functions);

let mut codes = CodeSection::new();
codes.function(&func);
module.section(&codes);

let wasm_bytes = module.finish();
Expand Down
2 changes: 1 addition & 1 deletion src/ir.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ pub use context::StepContext;
pub use event::Event;
pub use proc::{Proc, ProcMap, ProcRegistry, ProcedureContext};
pub use project::IrProject;
pub use step::{Step, RcStep};
pub use step::{RcStep, Step};
pub use target::Target;
pub use thread::Thread;
pub use types::{Type, TypeStack};
76 changes: 56 additions & 20 deletions src/ir/blocks.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::instructions::{fields, IrOpcode};
use crate::prelude::*;
use crate::sb3::{Block, BlockArray, BlockInfo, BlockMap, BlockOpcode};
use crate::sb3::{Block, BlockArray, BlockArrayOrId, BlockInfo, BlockMap, BlockOpcode, Input};
use fields::*;

pub fn from_block(block: &Block, blocks: &BlockMap) -> HQResult<Box<[IrOpcode]>> {
Expand All @@ -10,35 +10,71 @@ pub fn from_block(block: &Block, blocks: &BlockMap) -> HQResult<Box<[IrOpcode]>>
})
}

fn from_normal_block(block_info: &BlockInfo, _blocks: &BlockMap) -> HQResult<Box<[IrOpcode]>> {
Ok(match &block_info.opcode {
BlockOpcode::operator_add => [IrOpcode::operator_add].into_iter(),
BlockOpcode::looks_say => [IrOpcode::looks_say].into_iter(),
other => hq_todo!("unimplemented block: {:?}", other),
pub fn input_names(opcode: BlockOpcode) -> HQResult<Vec<String>> {
Ok(match opcode {
BlockOpcode::looks_say => vec!["MESSAGE"],
BlockOpcode::operator_add => vec!["NUM1", "NUM2"],
other => hq_todo!("unimplemented input_names for {:?}", other),
}
.into_iter()
.map(String::from)
.collect())
}

pub fn inputs(block_info: &BlockInfo, blocks: &BlockMap) -> HQResult<Vec<IrOpcode>> {
Ok(input_names(block_info.opcode.clone())?
.into_iter()
.map(|name| -> HQResult<Vec<IrOpcode>> {
match block_info
.inputs
.get((*name).into())
.ok_or(make_hq_bad_proj!("missing input {}", name))?
{
Input::NoShadow(_, Some(block)) | Input::Shadow(_, Some(block), _) => match block {
BlockArrayOrId::Array(arr) => Ok(vec![from_special_block(arr)?]),
BlockArrayOrId::Id(id) => match blocks
.get(id)
.ok_or(make_hq_bad_proj!("block for input {} doesn't exist", name))?
{
Block::Normal { block_info, .. } => {
Ok(from_normal_block(block_info, blocks)?.into())
}
Block::Special(block_array) => Ok(vec![from_special_block(block_array)?]),
},
},
_ => hq_bad_proj!("missing input block for {}", name),
}
})
.collect::<HQResult<Vec<_>>>()?
.iter()
.flatten()
.cloned()
.collect())
}

fn from_normal_block(block_info: &BlockInfo, blocks: &BlockMap) -> HQResult<Box<[IrOpcode]>> {
Ok(inputs(block_info, blocks)?
.into_iter()
.chain(match &block_info.opcode {
BlockOpcode::operator_add => [IrOpcode::operator_add].into_iter(),
BlockOpcode::looks_say => [IrOpcode::looks_say].into_iter(),
other => hq_todo!("unimplemented block: {:?}", other),
})
.collect())
}

fn from_special_block(block_array: &BlockArray) -> HQResult<IrOpcode> {
Ok(match block_array {
BlockArray::NumberOrAngle(ty, value) => match ty {
4 | 8 => IrOpcode::math_number(MathNumberFields(*value)),
5 => IrOpcode::math_positive_number(MathPositiveNumberFields(*value)),
6 => IrOpcode::math_whole_number(MathWholeNumberFields(*value as i64)),
7 => IrOpcode::math_integer(MathIntegerFields(*value as i64)),
4 | 5 | 8 => IrOpcode::hq_float(HqFloatFields(*value)),
6 | 7 => IrOpcode::hq_integer(HqIntegerFields(*value as i64)),
_ => hq_bad_proj!("bad project json (block array of type ({}, f64))", ty),
},
BlockArray::ColorOrString(ty, value) => match ty {
4 | 8 => IrOpcode::math_number(MathNumberFields(
value.parse().map_err(|_| make_hq_bug!(""))?,
)),
5 => IrOpcode::math_positive_number(MathPositiveNumberFields(
value.parse().map_err(|_| make_hq_bug!(""))?,
)),
6 => IrOpcode::math_whole_number(MathWholeNumberFields(
value.parse().map_err(|_| make_hq_bug!(""))?,
)),
7 => IrOpcode::math_integer(MathIntegerFields(
4 | 5 | 8 => {
IrOpcode::hq_float(HqFloatFields(value.parse().map_err(|_| make_hq_bug!(""))?))
}
6 | 7 => IrOpcode::hq_integer(HqIntegerFields(
value.parse().map_err(|_| make_hq_bug!(""))?,
)),
9 => hq_todo!(""),
Expand Down
7 changes: 6 additions & 1 deletion src/ir/proc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,12 @@ impl Proc {
step_context,
project,
)?,
None => RcStep::new(Rc::new(Step::new(None, step_context, Box::new([]), project))),
None => RcStep::new(Rc::new(Step::new(
None,
step_context,
Box::new([]),
project,
))),
};
Ok(Proc {
first_step,
Expand Down
43 changes: 22 additions & 21 deletions src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
#![cfg_attr(not(test), no_std)]
#![doc(html_logo_url = "https://hyperquark.github.io/hyperquark/logo.png")]
#![doc(html_favicon_url = "https://hyperquark.github.io/hyperquark/favicon.ico")]

#![allow(clippy::new_without_default)]

#[macro_use]
Expand All @@ -23,26 +22,6 @@ pub mod instructions;
#[doc(inline)]
pub use error::{HQError, HQErrorType, HQResult};

// use wasm::wasm;

#[cfg(target_family = "wasm")]
#[cfg_attr(target_family = "wasm", wasm_bindgen(js_namespace=console))]
extern "C" {
pub fn log(s: &str);
}

#[cfg(test)]
pub fn log(s: &str) {
println!("{s}")
}

// #[wasm_bindgen]
// pub fn sb3_to_wasm(proj: &str) -> Result<wasm::WasmProject, HQError> {
// let mut ir_proj = ir::IrProject::try_from(sb3::Sb3Project::try_from(proj)?)?;
// ir_proj.optimise()?;
// ir_proj.try_into()
// }

/// commonly used _things_ which would be nice not to have to type out every time
pub mod prelude {
pub use crate::{HQError, HQResult};
Expand All @@ -61,3 +40,25 @@ pub mod prelude {
pub type IndexMap<K, V> = indexmap::IndexMap<K, V, BuildHasherDefault<FNV1aHasher64>>;
pub type IndexSet<T> = indexmap::IndexSet<T, BuildHasherDefault<FNV1aHasher64>>;
}

use prelude::*;

// use wasm::wasm;

#[cfg(target_family = "wasm")]
#[cfg_attr(target_family = "wasm", wasm_bindgen(js_namespace=console))]
extern "C" {
pub fn log(s: &str);
}

#[cfg(test)]
pub fn log(s: &str) {
println!("{s}")
}

#[cfg_attr(target_family = "wasm", wasm_bindgen)]
pub fn sb3_to_wasm(proj: &str) -> HQResult<Vec<u8>> {
let sb3_proj = sb3::Sb3Project::try_from(proj)?;
let ir_proj: Rc<ir::IrProject> = sb3_proj.try_into()?;
wasm::WasmProject::try_from(ir_proj)?.finish()
}
Loading