Skip to content

Integrate VeloGraphX with BYO and define fair dynamic-container baselines #21

Description

@sauravsingla

Context

Brian Wheatman provided direct guidance on evaluating VeloGraphX through the BYO graph-container framework.

Key guidance:

  • Use the BYO paper/interface as the reference integration contract.
  • Baseline selection should favor the strongest relevant containers rather than an arbitrary broad set.
  • Batch Parallel CSR is a strong candidate baseline, subject to whether its workload assumptions match VeloGraphX.
  • Workload prioritization must be driven by the actual problem setting rather than a generic benchmark mix.
  • The evaluation must explicitly state whether it targets batch updates, concurrent updates/queries, parallel execution, and how many cores are used.
  • An upstream PR to the BYO repository is welcome if the VeloGraphX adapter is appropriate.

Reference interface:
https://github.com/wheatman/BYO/blob/00f8ead3925a3346d4322fd2b6f0b1ba1f1f9834/gbbs/graph.h#L2155

Candidate strong baseline:
https://brianwheatman.com/papers/batch_pcsr.pdf

Proposed work

  • Read and document the minimal BYO graph-container interface required for integration.
  • Implement a VeloGraphX storage adapter conforming to the BYO interface.
  • Keep the adapter storage-focused so algorithmic effects are not silently mixed into the comparison.
  • Identify the fastest relevant BYO dynamic containers for VeloGraphX's target semantics.
  • Include Batch Parallel CSR as a candidate baseline and justify inclusion/exclusion based on workload semantics.
  • Define whether the primary benchmark contract is batch-update, concurrent update/query, or both.
  • Define supported parallelism explicitly, including thread counts and machine topology.
  • Separate ingestion/update throughput, adjacency access, memory footprint, and analytical-kernel timing where possible.
  • Use identical datasets, update streams, thread counts, and correctness checks across comparable containers.
  • Report workloads where competing containers outperform VeloGraphX.
  • Document all configuration decisions and the expected performance consequences of each decision.
  • If the adapter is clean and generally useful, prepare an upstream PR to the BYO repository.

Experimental questions to answer

  1. How does VeloGraphX storage compare under pure batched updates?
  2. How does it compare when analytics run after each update batch?
  3. Is concurrent update/query support in scope for VeloGraphX, and if not, is that limitation explicit?
  4. How does performance change with core count?
  5. Which containers are strongest for the exact workload semantics VeloGraphX claims to support?
  6. Which observed gains come from storage layout versus the incremental analytics layer above it?

Acceptance criteria

This issue is complete when:

  1. A VeloGraphX BYO adapter exists and passes BYO correctness/interface checks.
  2. Baseline selection is justified using workload semantics and current performance, not convenience.
  3. Batch/concurrency assumptions and thread counts are explicit in the benchmark contract.
  4. Storage-level measurements are separated from VeloGraphX incremental-algorithm effects.
  5. At least one strong external dynamic-container baseline is reproduced under identical hardware and workload settings.
  6. Results include cases where the external baseline wins, if observed.
  7. The integration and reproduction steps are documented well enough for an external researcher to repeat.
  8. Upstream BYO contribution is attempted if the adapter is sufficiently general.

Goal

Use BYO as an independent apples-to-apples graph-container framework so VeloGraphX's storage claims can be evaluated separately from its dynamic analytics and adaptive execution policy.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions