Skip to content

Commit

Permalink
[Enhancement] Change LinkedListMultimap to ArrayListMultimap to bette…
Browse files Browse the repository at this point in the history
…r support for getting elements by index (StarRocks#11774)

When creating a table, FE will first put the tablet to be created into the TabletInvertedIndex and then send the task to BE to create the tablet. And when BE reports the tablet, FE will find the difference between the tablet reported by BE and the tablet in TabletInvertedIndex, at this time, if the bucket of table creation is relatively large, there will be a lot of extra tablet in TabletInvertedIndex. But in processing these extra tablet, FE put it into a LinkedList, and when getting the elements, it uses the get(int index) method, resulting in extremely inefficient. And this processing logic is put into the database write lock, resulting in the database stuck.
To fix this bug, we should use arrayList.
  • Loading branch information
gengjun-git authored Sep 28, 2022
1 parent 3d2fbe0 commit b06e52e
Showing 1 changed file with 6 additions and 6 deletions.
12 changes: 6 additions & 6 deletions fe/fe-core/src/main/java/com/starrocks/leader/ReportHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
package com.starrocks.leader;

import com.google.common.base.Preconditions;
import com.google.common.collect.LinkedListMultimap;
import com.google.common.collect.ArrayListMultimap;
import com.google.common.collect.ListMultimap;
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
Expand Down Expand Up @@ -313,23 +313,23 @@ private static void tabletReport(long backendId, Map<Long, TTablet> backendTable
GlobalStateMgr.getCurrentState().getPartitionIdToStorageMediumMap();

// db id -> tablet id
ListMultimap<Long, Long> tabletSyncMap = LinkedListMultimap.create();
ListMultimap<Long, Long> tabletSyncMap = ArrayListMultimap.create();
// db id -> tablet id
ListMultimap<Long, Long> tabletDeleteFromMeta = LinkedListMultimap.create();
ListMultimap<Long, Long> tabletDeleteFromMeta = ArrayListMultimap.create();
// tablet ids which schema hash is valid
Set<Long> foundTabletsWithValidSchema = new HashSet<Long>();
// tablet ids which schema hash is invalid
Map<Long, TTabletInfo> foundTabletsWithInvalidSchema = new HashMap<Long, TTabletInfo>();
// storage medium -> tablet id
ListMultimap<TStorageMedium, Long> tabletMigrationMap = LinkedListMultimap.create();
ListMultimap<TStorageMedium, Long> tabletMigrationMap = ArrayListMultimap.create();

// dbid -> txn id -> [partition info]
Map<Long, ListMultimap<Long, TPartitionVersionInfo>> transactionsToPublish = Maps.newHashMap();
Map<Long, Long> transactionsToCommitTime = Maps.newHashMap();
ListMultimap<Long, Long> transactionsToClear = LinkedListMultimap.create();
ListMultimap<Long, Long> transactionsToClear = ArrayListMultimap.create();

// db id -> tablet id
ListMultimap<Long, Long> tabletRecoveryMap = LinkedListMultimap.create();
ListMultimap<Long, Long> tabletRecoveryMap = ArrayListMultimap.create();

Set<Pair<Long, Integer>> tabletWithoutPartitionId = Sets.newHashSet();

Expand Down

0 comments on commit b06e52e

Please sign in to comment.