Skip to content

Unable to allocate 60.3 GiB for an array with shape (89997, 89997) and data type float64 #55

Open
@rybd111

Description

def composite_dis(df_wave, wave_all):
   dtw_dis = dict()
   for i in range(0, len(df_wave)):
        dtw_ij = dict()
        for j in range(0, len(df_wave[i])):
            krec = []
            for k in df_wave[i][j].keys():
                krec.append(k)
                for l in df_wave[i][j].keys():
                    if l not in krec:
                        if k == l:
                            continue
                        else:
                            wave_npk = np.empty( dtype=int, shape=[len(df_wave[i][j][k][k].values), 2] )
                            wave_npl = np.empty( dtype=int, shape=[len(df_wave[i][j][k][k].values), 2] )
                            wave_npk[:, 0] = df_wave[i][j][k]['data_time'].values
                            wave_npk[:, 1] = df_wave[i][j][k][k].values
                            wave_npl[:, 0] = df_wave[i][j][k]['data_time'].values
                            wave_npl[:, 1] = df_wave[i][j][l][l].values
                            
                            manhattan_distance = lambda x, y: np.abs(wave_npk[:, 1] - wave_npl[:, 1])
                            dtw_ij[k+'_'+l] = dtw(wave_npk[:, 1], wave_npl[:, 1], dist=manhattan_distance)
        dtw_dis[str(i)] = dtw_ij
    return dtw_dis`

"I use dtw package, but the np.array emerge the above problem."
what should I do?

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions