You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
keepDate takes a list of strings, with date component.
/** Removes all data that does not have selected date. * * @paramdates a list of dates * @paramcomponent the selected DateComponent enum value*/defkeepDate(
dates: List[String],
component: DateComponent=DateComponent.YYYYMMDD
):RDD[ArchiveRecord] = {
rdd.filter(r => dates.contains(ExtractDate(r.getCrawlDate, component)))
}
Keep dates, and discard dates are not consistent.
keepDate
takes a list of strings, with date component.discardDate
only takes a single date as a string.The text was updated successfully, but these errors were encountered: