Commit 9e77d48
[SPARK-21492][SQL][FOLLOW UP] Reimplement UnsafeExternalRowSorter in database style iterator
### What changes were proposed in this pull request?
Reimplement the iterator in UnsafeExternalRowSorter in database style. This can be done by reusing the `RowIterator` in our code base.
### Why are the changes needed?
During the job in #26164, after involving a var `isReleased` in `hasNext`, there's possible that `isReleased` is false when calling `hasNext`, but it becomes true before calling `next`. A safer way is using database-style iterator: `advanceNext` and `getRow`.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Existing UT.
Closes #26229 from xuanyuanking/SPARK-21492-follow-up.
Authored-by: Yuanjian Li <xyliyuanjian@gmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>1 parent 177bf67 commit 9e77d48
File tree
2 files changed
+26
-22
lines changed- sql
- catalyst/src/main/scala/org/apache/spark/sql/execution
- core/src/main/java/org/apache/spark/sql/execution
2 files changed
+26
-22
lines changedLines changed: 26 additions & 22 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
24 | 23 | | |
25 | 24 | | |
26 | 25 | | |
| |||
168 | 167 | | |
169 | 168 | | |
170 | 169 | | |
171 | | - | |
| 170 | + | |
172 | 171 | | |
173 | 172 | | |
174 | 173 | | |
175 | 174 | | |
176 | 175 | | |
177 | 176 | | |
178 | 177 | | |
179 | | - | |
| 178 | + | |
180 | 179 | | |
181 | 180 | | |
182 | 181 | | |
183 | 182 | | |
184 | 183 | | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | | - | |
| 184 | + | |
191 | 185 | | |
192 | | - | |
193 | | - | |
194 | | - | |
195 | | - | |
196 | | - | |
197 | | - | |
198 | | - | |
199 | | - | |
200 | | - | |
201 | | - | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
202 | 201 | | |
203 | | - | |
| 202 | + | |
| 203 | + | |
204 | 204 | | |
205 | 205 | | |
206 | 206 | | |
| |||
210 | 210 | | |
211 | 211 | | |
212 | 212 | | |
213 | | - | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
214 | 218 | | |
215 | 219 | | |
216 | 220 | | |
217 | 221 | | |
218 | 222 | | |
219 | 223 | | |
220 | | - | |
| 224 | + | |
221 | 225 | | |
222 | 226 | | |
223 | 227 | | |
| |||
0 commit comments