[SPARK-45024][PYTHON][CONNECT] Filter out some configurations in Sess…

…ion Creation

### What changes were proposed in this pull request?
apache#42694 filtered out static configurations in local mode

This filter out some configurations in both local mode and non-local mode, since the configurations are actually set after session creation, so configurations like `spark.remote` always take no effect.

### Why are the changes needed?
avoid unnecessary RPCs and warnings

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
CI

### Was this patch authored or co-authored using generative AI tooling?
no

Closes apache#42741 from zhengruifeng/filter_out_some_config.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>

python/pyspark/sql/connect/session.py

@@ -180,10 +180,16 @@ def enableHiveSupport(self) -> "SparkSession.Builder":
         def _apply_options(self, session: "SparkSession") -> None:
             with self._lock:
                 for k, v in self._options.items():
-                    try:
-                        session.conf.set(k, v)
-                    except Exception as e:
-                        warnings.warn(str(e))
+                    # the options are applied after session creation,
+                    # so following options always take no effect
+                    if k not in [
+                        "spark.remote",
+                        "spark.master",
+                    ]:
+                        try:
+                            session.conf.set(k, v)
+                        except Exception as e:
+                            warnings.warn(str(e))
 
         def create(self) -> "SparkSession":
             has_channel_builder = self._channel_builder is not None