Skip to content

Commit

Permalink
Update Spark.md
Browse files Browse the repository at this point in the history
  • Loading branch information
lanawwas authored Sep 10, 2023
1 parent b829811 commit 52644d1
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions Spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,15 @@ df = df.na.drop()
```

### Filter data using filter() command

#### Using a one-tenth of the data subset by calling filter() and using the rowID column
```python
filterDF = df.filter((df.rowID % 10) == 0) #Using a one-tenth of the data subset by calling filter() and using the rowID column
filterDF = df.filter((df.rowID % 10) == 0)
```
#### Filter data and count how many values are equal to 0.0

```python
filteredDF.filter(filteredDF.rain_accumulation == 0.0).count()
```
### Replace missing values with the column mean

```python
Expand Down

0 comments on commit 52644d1

Please sign in to comment.