Implement sbrk in MARSKernel

[pavelkryukov]

sbrk is a system call inside malloc. To run interesting programs, it should be supported inside MARSKernel class.

The idea is to keep pool of free heap memory in same manner as OS does.

[pavelkryukov]

@M-ximus Please provide scratch on interface for new class here.

[pavelkryukov]

Could you please share your expectation on completion?

[M-ximus]

I have some ideas about this:

1. Do we want manager that control all memory(something like real mmu) or control only high level operations(alloc, dealloc, access attributes)?
2. Two different model of memory works too different - implementation of plain memory can't allocate new memory, but hierarchic can via resize. How should I to resolve this conflict? May I change class HierarchiedMemory?

class MemoryManager
{
public:
MemoryManager(Addr ptr_to_mem_seg, uint64_t size);
void allocate_memory(int64_t num_bytes); //if it less than 0 it will decrease the segment
private:
// no need to define it now
};

[pavelkryukov]

Do we want manager that control all memory(something like real mmu) or control only high level operations(alloc, dealloc, access attributes)?

We definitely should start with high level operations. It would be nicer if the architecture you're making may be extended to add MMU features in future, but it is not necessary at the moment.

implementation of plain memory can't allocate new memory,

You're talking about host memory here, but we need to allocate guest memory, i.e. inside the FuncMemory class.

FuncMemory class models (on a functional level) a DRAM chip, and you cannot resize it dynamically. Instead, you need to provide pointers to free guest memory and do not provide pointers to allocated.

public:
MemoryManager(Addr ptr_to_mem_seg, uint64_t size);
void allocate_memory(int64_t num_bytes); //if it less than 0 it will decrease the segment

Please use our local types defined in <infra/types.h>. For size type, it's always better to use size_t.
How is pointer to allocated memory returned?
What is the interface for memory free and/or reallocation?

[M-ximus]

We can return pointer from Addr allocate_memory(size_t num_bytes);
I thought about functions like realloc() and free(). But I don't sure that we want C-style memory manager. Realloc() can be implemented via sbrk and searching in the table the most suitable space for this block.

Also void free(Addr mem_ptr) can be implemented via marking this space in the table(or linked list). But do we want to think about safety? Should we control attributes and/or attempts to access to not allocated memory(or memory of other application)?

[pavelkryukov]

But I don't sure that we want C-style memory manager

I cannot imagine any alternative, could you please explain?

But do we want to think about safety? Should we control attributes and/or attempts to access to not allocated memory(or memory of other application)?

Yes we should, but that is performed with TLB which is another issue (#625).

[M-ximus]

I thought that you wanted MMU(That translates virtual addresses to physical. We have plain and hierarchy model of memory that try to address memory and MMU tries to give them information from this address. It takes virtual address and via linkage table translate it to physical. But both of implementation of memory uses their own arrays for data). How should I work with this models? Can I change them?
If I should implement C-style memory manager, I will make hash table + linked list that will use existed memory classes. It will be simpler for using in applications but it's part of OS. But addresses that it will return will be an offset in array in memory class.(It controls only position of block in virtual memory and thinks about memory how unlimited(<= 2^64) storage that can return exception)
What type do you want?
Does my class be lower or higher of memory classes(hierarchy and plain)?

[pavelkryukov]

I thought that you wanted MMU(That translates virtual addresses to physical.

In MIPS/RISC-V translation is handled by TLB.

It will be simpler for using in applications but it's part of OS.

Surely. We are implementing MARS-OS kernel at the moment.

Does my class be lower or higher of memory classes(hierarchy and plain)?

I believe it should capture FuncMemory by pointer.

[M-ximus]

Should my class control more than one block of memory or only one(sbrk changes size of data segment(only one block))?

[pavelkryukov]

MARS does not operate with segmented memory, so there is no need right now.

[pavelkryukov]

However, we have stack, OS code and executable code loaded in memory. It worth to protect them from overwrite.

[M-ximus]

I used FuncMemoryReplicant as a basis:
class MemoryManager : public FuncMemory
{
public:
MemoryManager( Addr ptr_to_start_mem = 0, size_t mem_size = 4096 /*FIXME(M-ximus): Do we need a header with mem constants?*/);
Addr allocate( size_t num_bytes);// only checks (block_size <= max) and changes block_size. It returns pointer to start of new memory
size_t memcpy_guest_to_host( Byte* dst, Addr src, size_t size) const final;// checks (block_size <= max)
size_t memcpy_host_to_guest( Addr dst, const Byte* src, size_t size) final;// checks (block_size <= max)
size_t strlen( Addr addr) const final;
std::string dump() const final;
void duplicate_to( std::shared_ptr<WriteableMemory> target) const final;
private:
std::shared_ptr<FuncMemory> memory;
size_t block_size;
}

And sbrk will call allocate()

[pavelkryukov]

Please use the following syntax in your replies, otherwise it becomes to complicated.

```c++
class Code { };
```

[pavelkryukov]

class MemoryManager : public FuncMemory

The basic idea of inheritance is to represent "is" relation.
Additionally, FuncMemory is a class which supports two things: read and write, plus some convinient wrappers
So please think about the question: is MemoryManager a FuncMemory, i.e. is MemoryManager a thing which can be read or written?

[M-ximus]

Firstly, MemoryManager a thing that control read(load) and write(store). And other operations(like memcpy) need to use them. But this class should use memory that uses memcpy(while(true)). But it can control only access to memory(wrapper of FuncMemory). It should have safe read and write that can control all operations with memory.

class MemoryManager
{
public:
    MemoryManager( Addr ptr_to_start_mem = 0, size_t mem_size = 4096 /*FIXME(M-ximus): Do we need a header with mem constants?*/);
    Addr allocate( size_t num_bytes);// only checks (block_size <= max) and changes block_size. It returns pointer to start of new memory
    template<typename T, Endian endian> T load( Addr addr) const noexcept;
    std::string read_string( Addr addr) const;
    // I don't sure that we should be able to use unsafe read
    template<typename T, Endian endian> void store( T value, Addr addr);
    void write_string( const std::string& value, Addr addr);
    // I don't sure that we should be able to use unsafe write
    std::string dump();
private:
    std::shared_ptr<FuncMemory> memory;
    size_t block_size;
    Addr start_ptr;
}

(other methods will be added if they will be needed)
And I want to use object of this class in BaseKernel instead of FuncMemoryReplicant

[pavelkryukov]

MemoryManager a thing that control read(load) and write(store)

Nope. You can control address/page permissions without accessing real memory, by checking addresses in TLB or MemoryManager

I believe you need that:

class MemoryManager
{
public:
    explicit MemoryManager( std::shared_ptr<FuncMemory>);
    Addr allocate( size_t num_bytes);
    void deallocate( Addr addr);

    // Optional
    bool is_readable( Addr addr) const;
    bool is_writeable( Addr addr) const;
private:
    std::shared_ptr<FuncMemory> memory;
    // ...
};

[pavelkryukov]

Since we are aligned on interfaces, let's proceed to the next step — unit tests for the interface.

[pavelkryukov]

Could you please share your expectation on completion?

The question is still actual, although unanswered.

[pavelkryukov]

Removing back to pool since there is no response from owner