Skip to content

feat(blockifier): add native execution engine #620

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 77 commits into from
Oct 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
6ea8ef1
refactor: split syscalls into separate files
varex83 Aug 6, 2024
5b4cc97
feat: add ability of native execution
varex83 Aug 8, 2024
b254717
fix: make comments capitalized
varex83 Aug 8, 2024
024ffb6
fix: apply fmt
varex83 Aug 8, 2024
a54282c
fix: small fixes in comments and matching
varex83 Aug 8, 2024
7913859
fix: bring back old `.toml` formatting
varex83 Aug 8, 2024
2853de4
fix: remove unused yet utility functions
varex83 Aug 8, 2024
070d6d6
chore: update Cargo.lock
varex83 Aug 16, 2024
49d9ec7
feat: update Cargo.lock
varex83 Aug 19, 2024
79db01b
fix: address review
varex83 Aug 20, 2024
3b2d7be
refactor: address `get_entry_point` review requests
varex83 Aug 20, 2024
e4fbcbb
refactor: move out methods that won't be reviewed in this PR
varex83 Aug 20, 2024
5319302
fix: rust fmt
varex83 Aug 20, 2024
c3e2d7e
fix: apply some of the CI fixes
varex83 Aug 20, 2024
ce75c13
fix: comment in Cargo.toml
varex83 Aug 21, 2024
86f34b2
fix: address some of the review comments
varex83 Aug 27, 2024
7b7b825
refactor: define the type for lookup hashmap
varex83 Sep 9, 2024
3e2c16c
fix: remove Clone trait from call info, remove fallback-related stuff
varex83 Sep 13, 2024
f195de7
fix: ci
varex83 Aug 29, 2024
ecf4f6f
fix: return back old images in the CI
varex83 Aug 29, 2024
4cf1088
chore: update feature contracts
varex83 Aug 29, 2024
271ff7a
fix: add LLVM deps to the Dockerfile
varex83 Aug 29, 2024
764ade8
fix: add LLVM to `install_build_tools.sh`
varex83 Aug 30, 2024
105d963
fix: update build tools, add env vars
varex83 Aug 30, 2024
59e66a0
fix: add clean up of the native artifacts
varex83 Sep 2, 2024
e6501a6
fix: add clean up of the native artifacts
varex83 Sep 2, 2024
d08a1ea
fix: native artifacts push CI
varex83 Sep 3, 2024
48b775f
fix: add env to ignore interactive env
varex83 Sep 3, 2024
8ac91b1
fix: update sequencer-ci.Dockerfile
varex83 Sep 3, 2024
bf69200
fix: update sequencer-ci.Dockerfile
varex83 Sep 3, 2024
5627a84
fix: try to fix Dockerfile with adding musl-g++
varex83 Sep 3, 2024
bd3a7ce
fix: try to fix Dockerfile with adding musl-g++ #2
varex83 Sep 3, 2024
1c5549a
fix: try to fix Dockerfile with adding musl-g++ #2
varex83 Sep 3, 2024
7eea141
fix: try to fix Dockerfile with adding musl-g++ #4
varex83 Sep 3, 2024
6e660e3
fix: try to fix Dockerfile with adding musl-g++ #4
varex83 Sep 3, 2024
12e96a8
fix: try to fix Dockerfile
varex83 Sep 3, 2024
9f6c221
fix: try to fix Dockerfile finally?
varex83 Sep 3, 2024
5f36ab0
fix: try to fix Dockerfile
varex83 Sep 3, 2024
2b99646
fix: add hard option for sym link creation
varex83 Sep 3, 2024
6e8d246
fix: make Dockerfile dynamic
varex83 Sep 5, 2024
5405222
fix: script
varex83 Sep 11, 2024
ec80cde
fix: script
varex83 Sep 11, 2024
7ce6470
fix: verify_cairo_file_dependencies ci
varex83 Sep 11, 2024
4552775
fix: cairo native runtime library path
varex83 Sep 16, 2024
a6377b5
fix: conflicts after merge
varex83 Sep 16, 2024
efbe451
fix: update feature contracts
varex83 Sep 16, 2024
2690010
chore: update Cargo.lock
varex83 Sep 17, 2024
d976b02
fix: conflicts after merge
varex83 Sep 17, 2024
00eca46
fix: address review comments
varex83 Sep 17, 2024
767924f
fix: fix review comment regarding `function_idx`
varex83 Sep 17, 2024
805ec8f
refactor: lookup table to use `HashMap` instead of `Vec`
varex83 Sep 18, 2024
a2a6ebe
refactor: make `verify_constructor` be part of `CallEntryPoint`
varex83 Sep 18, 2024
e11dda0
fix: add panic instead of todo!
varex83 Sep 18, 2024
fba9a70
fix: update Cargo.lock
varex83 Sep 18, 2024
8ddf32f
fix: ci
varex83 Sep 18, 2024
50567d6
fix: address review comments
varex83 Sep 19, 2024
7be64d7
fix: address review comments
varex83 Sep 20, 2024
70c9f83
fix: crate name mismatch
rodrigo-pino Oct 1, 2024
b164f96
chore: update Cargo.lock
rodrigo-pino Oct 2, 2024
74cab55
fix: compilation errors
rodrigo-pino Oct 2, 2024
206cc93
fix: cargo missing crate paths
rodrigo-pino Oct 2, 2024
4fa3885
chore: fmt
rodrigo-pino Oct 2, 2024
be3265d
fix: cargo docs
rodrigo-pino Oct 3, 2024
9cf34a5
fix: remove old setup-native-deps
rodrigo-pino Oct 3, 2024
c501572
chore: reapply format :p
rodrigo-pino Oct 3, 2024
38629c7
chore: remove transaction_utils.rs
rodrigo-pino Oct 3, 2024
8ca2e75
chore: swap string delimeters in .ymls
rodrigo-pino Oct 6, 2024
c3e679b
fix: add zstd to Blockifier image
rodrigo-pino Oct 6, 2024
e670abf
refactor: remove manual dependency installation in Dockerfile
rodrigo-pino Oct 6, 2024
34d77b4
chore: restore committer_ci.yml
rodrigo-pino Oct 6, 2024
447948e
chore: restore papyrus_nightly-tests.yml
rodrigo-pino Oct 6, 2024
3b91c09
chore: restore sequencer-ci.Dockerfile
rodrigo-pino Oct 6, 2024
b0397dc
refactor: panics -> todo & new msgs for Native Class
rodrigo-pino Oct 6, 2024
38fdfbe
chore: restore blockifier_compiled_cairo.yml, blockifier_post-merge.y…
rodrigo-pino Oct 6, 2024
31b9cec
fix: re-import BigUintAsHex
rodrigo-pino Oct 7, 2024
9a9e4bb
chore: restore commiter_ci.yml and test_contract_casm.json
rodrigo-pino Oct 7, 2024
2986be6
chore: update Cargo.lock
rodrigo-pino Oct 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
349 changes: 192 additions & 157 deletions Cargo.lock

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions crates/blockifier/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ ark-secp256r1.workspace = true
cached.workspace = true
cairo-lang-casm = { workspace = true, features = ["parity-scale-codec"] }
cairo-lang-runner.workspace = true
cairo-lang-sierra.workspace = true
cairo-lang-starknet-classes.workspace = true
cairo-lang-utils.workspace = true
cairo-native.workspace = true
Expand Down
1 change: 1 addition & 0 deletions crates/blockifier/src/execution.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,6 @@ pub mod entry_point_execution;
pub mod errors;
pub mod execution_utils;
pub mod hint_code;
pub mod native;
pub mod stack_trace;
pub mod syscalls;
181 changes: 169 additions & 12 deletions crates/blockifier/src/execution/contract_class.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
use std::collections::{HashMap, HashSet};
use std::ops::Deref;
use std::ops::{Deref, Index};
use std::sync::Arc;

use cairo_lang_casm;
use cairo_lang_casm::hints::Hint;
use cairo_lang_sierra::ids::FunctionId;
use cairo_lang_starknet_classes::casm_contract_class::{CasmContractClass, CasmContractEntryPoint};
use cairo_lang_starknet_classes::contract_class::{
ContractClass as SierraContractClass,
ContractEntryPoint as SierraContractEntryPoint,
};
use cairo_lang_starknet_classes::NestedIntList;
use cairo_lang_utils::bigint::BigUintAsHex;
#[allow(unused_imports)]
use cairo_native::executor::AotNativeExecutor;
use cairo_vm::serde::deserialize_program::{
Expand Down Expand Up @@ -34,12 +40,11 @@ use starknet_api::deprecated_contract_class::{
};
use starknet_types_core::felt::Felt;

use super::execution_utils::poseidon_hash_many_cost;
use crate::abi::abi_utils::selector_from_name;
use crate::abi::constants::{self, CONSTRUCTOR_ENTRY_POINT_NAME};
use crate::abi::constants::{self};
use crate::execution::entry_point::CallEntryPoint;
use crate::execution::errors::{ContractClassError, PreExecutionError};
use crate::execution::execution_utils::sn_api_to_cairo_vm_program;
use crate::execution::execution_utils::{poseidon_hash_many_cost, sn_api_to_cairo_vm_program};
use crate::execution::native::utils::contract_entrypoint_to_entrypoint_selector;
use crate::fee::eth_gas_constants;
use crate::transaction::errors::TransactionExecutionError;
use crate::versioned_constants::CompilerVersion;
Expand All @@ -64,6 +69,7 @@ pub enum TrackedResource {
pub enum ContractClass {
V0(ContractClassV0),
V1(ContractClassV1),
V1Native(NativeContractClassV1),
}

impl TryFrom<RawContractClass> for ContractClass {
Expand All @@ -88,13 +94,17 @@ impl ContractClass {
match self {
ContractClass::V0(class) => class.constructor_selector(),
ContractClass::V1(class) => class.constructor_selector(),
ContractClass::V1Native(class) => class.constructor_selector(),
}
}

pub fn estimate_casm_hash_computation_resources(&self) -> ExecutionResources {
match self {
ContractClass::V0(class) => class.estimate_casm_hash_computation_resources(),
ContractClass::V1(class) => class.estimate_casm_hash_computation_resources(),
ContractClass::V1Native(_) => {
todo!("Use casm to estimate casm hash computation resources")
}
}
}

Expand All @@ -107,13 +117,19 @@ impl ContractClass {
panic!("get_visited_segments is not supported for v0 contracts.")
}
ContractClass::V1(class) => class.get_visited_segments(visited_pcs),
ContractClass::V1Native(_) => {
panic!("get_visited_segments is not supported for native contracts.")
}
}
}

pub fn bytecode_length(&self) -> usize {
match self {
ContractClass::V0(class) => class.bytecode_length(),
ContractClass::V1(class) => class.bytecode_length(),
ContractClass::V1Native(_) => {
todo!("implement bytecode_length for native contracts.")
}
}
}

Expand All @@ -124,6 +140,7 @@ impl ContractClass {
ContractClass::V1(contract_class) => {
contract_class.tracked_resource(min_sierra_version)
}
ContractClass::V1Native(_) => TrackedResource::SierraGas,
}
}
}
Expand Down Expand Up @@ -237,11 +254,7 @@ impl ContractClassV1 {
&self,
call: &CallEntryPoint,
) -> Result<EntryPointV1, PreExecutionError> {
if call.entry_point_type == EntryPointType::Constructor
&& call.entry_point_selector != selector_from_name(CONSTRUCTOR_ENTRY_POINT_NAME)
{
return Err(PreExecutionError::InvalidConstructorEntryPointName);
}
call.verify_constructor()?;

let entry_points_of_same_type = &self.0.entry_points_by_type[&call.entry_point_type];
let filtered_entry_points: Vec<_> = entry_points_of_same_type
Expand Down Expand Up @@ -503,7 +516,7 @@ pub fn deserialize_program<'de, D: Deserializer<'de>>(
// V1 utilities.

// TODO(spapini): Share with cairo-lang-runner.
fn hint_to_hint_params(hint: &cairo_lang_casm::hints::Hint) -> Result<HintParams, ProgramError> {
fn hint_to_hint_params(hint: &Hint) -> Result<HintParams, ProgramError> {
Ok(HintParams {
code: serde_json::to_string(hint)?,
accessible_scopes: vec![],
Expand Down Expand Up @@ -582,7 +595,7 @@ impl ClassInfo {
) -> ContractClassResult<Self> {
let (contract_class_version, condition) = match contract_class {
ContractClass::V0(_) => (0, sierra_program_length == 0),
ContractClass::V1(_) => (1, sierra_program_length > 0),
ContractClass::V1(_) | ContractClass::V1Native(_) => (1, sierra_program_length > 0),
};

if condition {
Expand All @@ -595,3 +608,147 @@ impl ClassInfo {
}
}
}

// Cairo-native utilities.

#[derive(Clone, Debug, PartialEq, Eq)]
pub struct NativeContractClassV1(pub Arc<NativeContractClassV1Inner>);
impl Deref for NativeContractClassV1 {
type Target = NativeContractClassV1Inner;

fn deref(&self) -> &Self::Target {
&self.0
}
}

impl NativeContractClassV1 {
fn constructor_selector(&self) -> Option<EntryPointSelector> {
self.entry_points_by_type.constructor.first().map(|ep| ep.selector)
}

/// Initialize a compiled contract class for native.
///
/// executor must be derived from sierra_program which in turn must be derived from
/// sierra_contract_class.
pub fn new(
executor: AotNativeExecutor,
sierra_contract_class: SierraContractClass,
) -> NativeContractClassV1 {
let contract = NativeContractClassV1Inner::new(executor, sierra_contract_class);

Self(Arc::new(contract))
}

/// Returns an entry point into the natively compiled contract.
pub fn get_entry_point(&self, call: &CallEntryPoint) -> Result<&FunctionId, PreExecutionError> {
call.verify_constructor()?;

let entry_points_of_same_type = &self.0.entry_points_by_type[call.entry_point_type];
let filtered_entry_points: Vec<_> = entry_points_of_same_type
.iter()
.filter(|ep| ep.selector == call.entry_point_selector)
.collect();

match &filtered_entry_points[..] {
[] => Err(PreExecutionError::EntryPointNotFound(call.entry_point_selector)),
[entry_point] => Ok(&entry_point.function_id),
_ => Err(PreExecutionError::DuplicatedEntryPointSelector {
selector: call.entry_point_selector,
typ: call.entry_point_type,
}),
}
}
}

#[derive(Debug)]
pub struct NativeContractClassV1Inner {
pub executor: AotNativeExecutor,
entry_points_by_type: NativeContractEntryPoints,
// Storing the raw sierra program and entry points to be able to compare the contract class
sierra_program: Vec<BigUintAsHex>,
}

impl NativeContractClassV1Inner {
fn new(executor: AotNativeExecutor, sierra_contract_class: SierraContractClass) -> Self {
NativeContractClassV1Inner {
executor,
entry_points_by_type: NativeContractEntryPoints::from(&sierra_contract_class),
sierra_program: sierra_contract_class.sierra_program,
}
}
}

// The location where the compiled contract is loaded into memory will not
// be the same therefore we exclude it from the comparison.
impl PartialEq for NativeContractClassV1Inner {
fn eq(&self, other: &Self) -> bool {
self.entry_points_by_type == other.entry_points_by_type
&& self.sierra_program == other.sierra_program
}
}

impl Eq for NativeContractClassV1Inner {}

#[derive(Debug, PartialEq)]
/// Modelled after [cairo_lang_starknet_classes::contract_class::ContractEntryPoints]
/// and enriched with information for the Cairo Native ABI.
struct NativeContractEntryPoints {
constructor: Vec<NativeEntryPoint>,
external: Vec<NativeEntryPoint>,
l1_handler: Vec<NativeEntryPoint>,
}

impl From<&SierraContractClass> for NativeContractEntryPoints {
fn from(sierra_contract_class: &SierraContractClass) -> Self {
let program =
sierra_contract_class.extract_sierra_program().expect("Can't get sierra program.");

let func_ids = program.funcs.iter().map(|func| &func.id).collect::<Vec<&FunctionId>>();

let entry_points_by_type = &sierra_contract_class.entry_points_by_type;

NativeContractEntryPoints {
constructor: sierra_eps_to_native_eps(&func_ids, &entry_points_by_type.constructor),
external: sierra_eps_to_native_eps(&func_ids, &entry_points_by_type.external),
l1_handler: sierra_eps_to_native_eps(&func_ids, &entry_points_by_type.l1_handler),
}
}
}

impl Index<EntryPointType> for NativeContractEntryPoints {
type Output = Vec<NativeEntryPoint>;

fn index(&self, index: EntryPointType) -> &Self::Output {
match index {
EntryPointType::Constructor => &self.constructor,
EntryPointType::External => &self.external,
EntryPointType::L1Handler => &self.l1_handler,
}
}
}

fn sierra_eps_to_native_eps(
func_ids: &[&FunctionId],
sierra_eps: &[SierraContractEntryPoint],
) -> Vec<NativeEntryPoint> {
sierra_eps.iter().map(|sierra_ep| NativeEntryPoint::from(func_ids, sierra_ep)).collect()
}

#[derive(Debug, PartialEq)]
/// Provides a relation between a function in a contract and a compiled contract.
struct NativeEntryPoint {
/// The selector is the key to find the function in the contract.
selector: EntryPointSelector,
/// And the function_id is the key to find the function in the compiled contract.
function_id: FunctionId,
}

impl NativeEntryPoint {
fn from(func_ids: &[&FunctionId], sierra_ep: &SierraContractEntryPoint) -> NativeEntryPoint {
let &function_id = func_ids.get(sierra_ep.function_idx).expect("Can't find function id.");
NativeEntryPoint {
selector: contract_entrypoint_to_entrypoint_selector(sierra_ep),
function_id: function_id.clone(),
}
}
}
10 changes: 10 additions & 0 deletions crates/blockifier/src/execution/entry_point.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ use starknet_types_core::felt::Felt;

use crate::abi::abi_utils::selector_from_name;
use crate::abi::constants;
use crate::abi::constants::CONSTRUCTOR_ENTRY_POINT_NAME;
use crate::context::{BlockContext, TransactionContext};
use crate::execution::call_info::CallInfo;
use crate::execution::common_hints::ExecutionMode;
Expand Down Expand Up @@ -173,6 +174,15 @@ impl CallEntryPoint {

execution_result
}
pub fn verify_constructor(&self) -> Result<(), PreExecutionError> {
if self.entry_point_type == EntryPointType::Constructor
&& self.entry_point_selector != selector_from_name(CONSTRUCTOR_ENTRY_POINT_NAME)
{
Err(PreExecutionError::InvalidConstructorEntryPointName)
} else {
Ok(())
}
}
}

pub struct ConstructorContext {
Expand Down
2 changes: 1 addition & 1 deletion crates/blockifier/src/execution/errors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ impl From<RunnerError> for PreExecutionError {
#[derive(Debug, Error)]
pub enum PostExecutionError {
#[error(transparent)]
MathError(#[from] cairo_vm::types::errors::math_errors::MathError),
MathError(#[from] MathError),
#[error(transparent)]
MemoryError(#[from] MemoryError),
#[error(transparent)]
Expand Down
3 changes: 3 additions & 0 deletions crates/blockifier/src/execution/execution_utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,9 @@ pub fn execute_entry_point_call(
resources,
context,
),
ContractClass::V1Native(_contract_class) => {
unimplemented!("Native contract entry point execution is not yet implemented.")
}
}
}

Expand Down
1 change: 1 addition & 0 deletions crates/blockifier/src/execution/native.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pub mod utils;
9 changes: 9 additions & 0 deletions crates/blockifier/src/execution/native/utils.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
use cairo_lang_starknet_classes::contract_class::ContractEntryPoint;
use starknet_api::core::EntryPointSelector;
use starknet_types_core::felt::Felt;

pub fn contract_entrypoint_to_entrypoint_selector(
entrypoint: &ContractEntryPoint,
) -> EntryPointSelector {
EntryPointSelector(Felt::from(&entrypoint.selector))
}
2 changes: 1 addition & 1 deletion crates/blockifier/src/execution/syscalls/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -524,7 +524,7 @@ pub fn replace_class(
ContractClass::V0(_) => {
Err(SyscallExecutionError::ForbiddenClassReplacement { class_hash })
}
ContractClass::V1(_) => {
ContractClass::V1(_) | ContractClass::V1Native(_) => {
syscall_handler
.state
.set_class_hash_at(syscall_handler.storage_address(), class_hash)?;
Expand Down
3 changes: 3 additions & 0 deletions crates/blockifier/src/test_utils/contracts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,9 @@ impl FeatureContract {
.unwrap()
.offset
}
ContractClass::V1Native(_) => {
panic!("Not implemented for cairo native contracts")
}
}
}

Expand Down
2 changes: 1 addition & 1 deletion crates/blockifier/src/transaction/account_transaction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -931,7 +931,7 @@ impl ValidatableTransaction for AccountTransaction {

// Validate return data.
let contract_class = state.get_compiled_contract_class(class_hash)?;
if let ContractClass::V1(_) = contract_class {
if matches!(contract_class, ContractClass::V1(_) | ContractClass::V1Native(_)) {
// The account contract class is a Cairo 1.0 contract; the `validate` entry point should
// return `VALID`.
let expected_retdata = retdata![Felt::from_hex(constants::VALIDATE_RETDATA)?];
Expand Down
2 changes: 1 addition & 1 deletion crates/blockifier/src/transaction/test_utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -369,7 +369,7 @@ fn create_all_resource_bounds(
pub fn calculate_class_info_for_testing(contract_class: ContractClass) -> ClassInfo {
let sierra_program_length = match contract_class {
ContractClass::V0(_) => 0,
ContractClass::V1(_) => 100,
ContractClass::V1(_) | ContractClass::V1Native(_) => 100,
};
ClassInfo::new(&contract_class, sierra_program_length, 100).unwrap()
}
Expand Down
2 changes: 1 addition & 1 deletion crates/blockifier/src/transaction/transactions.rs
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ impl DeclareTransaction {
})?
}
}
ContractClass::V1(_) => {
ContractClass::V1(_) | ContractClass::V1Native(_) => {
if declare_version <= TransactionVersion::ONE {
Err(TransactionExecutionError::ContractClassVersionMismatch {
declare_version,
Expand Down
3 changes: 2 additions & 1 deletion scripts/dependencies.sh
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ function setup_llvm_deps() {
function main() {
[ "$(uname)" = "Linux" ] && install_essential_deps_linux
setup_llvm_deps
echo "LLVM and Cairo native runtime dependencies installed successfully."
echo "LLVM dependencies installed successfully."
}

main "$@"

Loading