Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to use deep neural network correctly #313

Open
procodingjob opened this issue Dec 28, 2024 · 0 comments
Open

How to use deep neural network correctly #313

procodingjob opened this issue Dec 28, 2024 · 0 comments

Comments

@procodingjob
Copy link

procodingjob commented Dec 28, 2024

first of all thank you for this good project and your great effort

how i can use deep neural network correctlt ?
also how we can set layers for projection and prediction ?

i try this : [512, 256, 128, 64, 32] but is like getting same number of neuron on each layer !!

policy=dict(
    
    model=dict(
        observation_shape=observation_shape,
        action_space_size=action_space_size,
        model_type='mlp',
        lstm_hidden_size=1024,
        latent_state_dim=1024,

        fc_reward_layers=[512, 256, 128, 64, 32],  
        fc_value_layers=[512, 256, 128, 64, 32],   
        fc_policy_layers=[512, 256, 128, 64, 32],  

        self_supervised_learning_loss=True,  # NOTE: default is False.
        discrete_action_encoding_type='one_hot',
        norm_type='BN',
    ),
    


  **see the output please** 
  
  
                  MuZeroModelMLP(                                                                                                                                                                                                            
                      (representation_network): RepresentationNetworkMLP(                                                                                                                                                                      
                        (fc_representation): Sequential(                                                                                                                                                                                       
                          (0): Linear(in_features=120, out_features=1024, bias=True)                                                                                                                                                           
                          (1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (2): GELU(approximate='tanh')                                                                                                                                                                                        
                          (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                        )                                                                                                                                                                                                                      
                        (sim_norm): SimNorm(dim=8)                                                                                                                                                                                             
                      )                                                                                                                                                                                                                        
                      (dynamics_network): DynamicsNetwork(                                                                                                                                                                                     
                        (fc_dynamics_1): Sequential(                                                                                                                                                                                           
                          (0): Linear(in_features=1027, out_features=1024, bias=True)                                                                                                                                                          
                          (1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (4): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                        )                                                                                                                                                                                                                      
                        (fc_dynamics_2): Sequential(                                                                                                                                                                                           
                          (0): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (4): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                        )                                                                                                                                                                                                                      
                        (fc_reward_head): Sequential(                                                                                                                                                                                          
                          (0): Linear(in_features=1024, out_features=512, bias=True)                                                                                                                                                           
                          (1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=512, out_features=601, bias=True)                                                                                                                                                            
                        )                                                                                                                                                                                                                      
                      )                                                                                                                                                                                                                        
                      (prediction_network): PredictionNetworkMLP(                                                                                                                                                                              
                        (fc_prediction_common): Sequential(                                                                                                                                                                                    
                          (0): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                          
                          (4): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                        )                                                                                                                                                                                                                      
                        (fc_value_head): Sequential(                                                                                                                                                                                           
                          (0): Linear(in_features=1024, out_features=512, bias=True)                                                                                                                                                           
                          (1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (4): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                          (6): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (7): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (8): ReLU(inplace=True)                                                                                                                                                                                              
                          (9): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (10): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (11): ReLU(inplace=True)                                                                                                                                                                                             
                          (12): Linear(in_features=512, out_features=601, bias=True)                                                                                                                                                           
                        )                                                                                                                                                                                                                      
                        (fc_policy_head): Sequential(                                                                                                                                                                                          
                          (0): Linear(in_features=1024, out_features=512, bias=True)                                                                                                                                                           
                          (1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (2): ReLU(inplace=True)                                                                                                                                                                                              
                          (3): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (4): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (5): ReLU(inplace=True)                                                                                                                                                                                              
                          (6): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (7): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                           
                          (8): ReLU(inplace=True)                                                                                                                                                                                              
                          (9): Linear(in_features=512, out_features=512, bias=True)                                                                                                                                                            
                          (10): LayerNorm((512,), eps=1e-05, elementwise_affine=True)                                                                                                                                                          
                          (11): ReLU(inplace=True)                                                                                                                                                                                             
                          (12): Linear(in_features=512, out_features=3, bias=True)                                                                                                                                                             
                        )                                                                                                                                                                                                                      
                      )                                                                                                                                                                                                                        
                      (projection): Sequential(                                                                                                                                                                                                
                        (0): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                            
                        (1): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)                                                                                                                                 
                        (2): ReLU(inplace=True)                                                                                                                                                                                                
                        (3): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                            
                        (4): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)                                                                                                                                 
                        (5): ReLU(inplace=True)                                                                                                                                                                                                
                        (6): Linear(in_features=1024, out_features=1024, bias=True)                                                                                                                                                            
                        (7): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)                                                                                                                                 
                      )                                                                                                                                                                                                                        
                      (prediction_head): Sequential(                                                                                                                                                                                           
                        (0): Linear(in_features=1024, out_features=512, bias=True)                                                                                                                                                             
                        (1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)                                                                                                                                  
                        (2): ReLU(inplace=True)                                                                                                                                                                                                
                        (3): Linear(in_features=512, out_features=1024, bias=True)                                                                                                                                                             
                      )                                                                                                                                                                                                                        
                    )  

thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant