Proposal: Integrate `arceos_posix_api` into this repository to enhance maintainability

[AsakuraMizu]

Background

I was trying to implement a few system calls and found that we now have a messy split between starry-next and arceos_posix_api.

Let's say, ppoll. I wanted to implement it in arceos_posix_api at first, because this crate already has similar functions such as select and epoll. But then I noticed that ppoll also needs to handle signals. Although we don't have a signal mechanism now, if we are going to implement it in starry-next in the future, then ppoll is best implemented in starry-next (or even only here). Also note that since select is implemented in arceos_posix_api, it would be tricky to implement pselect later.

Proposal

Currently, we have some syscalls that are proxied to arceos_posix_api functions, mainly file system related parts. This causes some maintenance inconveniences, for example, some syscalls depend on our TaskExt, others require the internal structure of arceos_posix_api, and some may require both. As a result, when implementing new features, you may need to modify both upstream and downstream repositories simultaneously.

My proposal is that we no longer rely on the arceos_posix_api crate, and maintain a separate syscall interface crate in this repository. This does not mean a complete rewrite, the existing code in arceos_posix_api can be ported and improved, and then other functions implemented for the monolithic kernel can be added.

Benefits

• More convenient development - Single source for syscall implementations, developing in a single repository
• Reduce duplication of functionality - We now repeat the same interface as arceos_posix_api in starry-next.
• More secure - Although we have UserPtr to check the safety of the user address, the raw pointer is still passed to the implementation instead of the safe Rust-style wrapper. And for the case of writev with nested pointers, we currently do not check it.

Implementation details

No changes to upstream repositories are required. We can divide starry-next into the following three crates:

• starry-core: mainly contains the current mm.rs, ptr.rs and task.rs, providing core functionality in the monolithic kernel
• starry-api: the current syscall_imp module merged with arceos_posix_api, where arceos_posix_api can enjoy the convenience brought by syscall_instrument or UserPtr when porting
• starry (already exists): entry main.rs and syscall handler handle_syscall

Thoughts

This is just a preliminary idea and may have some imperfections, so discussion and suggestions are welcome. Even if this proposal is not adopted, I still strongly recommend that we at least solve this issue to reduce the pain of future development.

[Azure-stars]

We originally hoped that in the future, support for more syscalls could be gradually merged into arceos_posix_api, and use mechanisms such as namespace to achieve a more elegant unification of macro kernels and unikernels, that is, the same code meets the needs of different architectures.

However, this requirement is indeed too ideal, so it is currently a feasible way to implement syscall directly in starry-next. But I don’t think it is necessary to completely remove the dependency on arceos_posix_api. Some already implemented functions (such as syscall_times) can be selectively reused.

The principle is: reuse existing content, but if there is new logic, implement it in starry-next.

[Azure-stars]

Background

I was trying to implement a few system calls and found that we now have a messy split between starry-next and arceos_posix_api.

Let's say, ppoll. I wanted to implement it in arceos_posix_api at first, because this crate already has similar functions such as select and epoll. But then I noticed that ppoll also needs to handle signals. Although we don't have a signal mechanism now, if we are going to implement it in starry-next in the future, then ppoll is best implemented in starry-next (or even only here). Also note that since select is implemented in arceos_posix_api, it would be tricky to implement pselect later.

Proposal

Currently, we have some syscalls that are proxied to arceos_posix_api functions, mainly file system related parts. This causes some maintenance inconveniences, for example, some syscalls depend on our TaskExt, others require the internal structure of arceos_posix_api, and some may require both. As a result, when implementing new features, you may need to modify both upstream and downstream repositories simultaneously.

My proposal is that we no longer rely on the arceos_posix_api crate, and maintain a separate syscall interface crate in this repository. This does not mean a complete rewrite, the existing code in arceos_posix_api can be ported and improved, and then other functions implemented for the monolithic kernel can be added.

Benefits

• More convenient development - Single source for syscall implementations, developing in a single repository
• Reduce duplication of functionality - We now repeat the same interface as arceos_posix_api in starry-next.
• More secure - Although we have UserPtr to check the safety of the user address, the raw pointer is still passed to the implementation instead of the safe Rust-style wrapper. And for the case of writev with nested pointers, we currently do not check it.

Implementation details

No changes to upstream repositories are required. We can divide starry-next into the following three crates:

• starry-core: mainly contains the current mm.rs, ptr.rs and task.rs, providing core functionality in the monolithic kernel
• starry-api: the current syscall_imp module merged with arceos_posix_api, where arceos_posix_api can enjoy the convenience brought by syscall_instrument or UserPtr when porting
• starry (already exists): entry main.rs and syscall handler handle_syscall

Thoughts

This is just a preliminary idea and may have some imperfections, so discussion and suggestions are welcome. Even if this proposal is not adopted, I still strongly recommend that we at least solve this issue to reduce the pain of future development.

Maybe we don't need to split the starry-core and starry. It is a good idea to make the syscall processing layer a separate crate, so that we can explore its reuse in other kernels in the future. However, it does not need to be stored separately like arceos crates (such as page_table_multiarch), and can still be stored in this repository.

[AsakuraMizu]

However, it does not need to be stored separately like arceos crates (such as page_table_multiarch), and can still be stored in this repository.

Yep I meant using cargo workspaces.

[Azure-stars]

What suggestions do you have for how syscall imp should be merged with arceos_posix_api in the future, or whether to separate it from arceos_posix_api?

Our idea is to reduce redundant code as much as possible, so it is better to merge.

[Azure-stars]

You can try it by creating a PR and we talk about it?