Skip to content

upload: Enable File Upload for users. #1414

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
1 change: 1 addition & 0 deletions docs/hotkeys.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@
## Composing a Message
|Command|Key Combination|
| :--- | :---: |
|Upload file|<kbd>ctrl</kbd> + <kbd>o</kbd>|
|Cycle through recipient and content boxes|<kbd>tab</kbd>|
|Send a message|<kbd>ctrl</kbd> + <kbd>d</kbd> / <kbd>meta</kbd> + <kbd>enter</kbd>|
|Save current message as a draft|<kbd>meta</kbd> + <kbd>s</kbd>|
Expand Down
45 changes: 45 additions & 0 deletions tests/model/test_model.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import copy
import json
import os
import tempfile
from collections import OrderedDict
from copy import deepcopy
from typing import Any, List, Optional, Tuple
Expand Down Expand Up @@ -810,6 +812,49 @@ def test_send_stream_message(
req, self.controller
)

@pytest.mark.parametrize(
"file_name, upload_result, expected_result",
[
case(
"existing_file.txt",
Comment on lines +816 to +819
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see that we gain from using different file_name strings, when this variable is passed in as it is each time. Instead I'd suggest using a fixed string - if we continue to set a filename to create - and then using an alternative parameter here which may have a different path. That would allow using the same branch of the conditional each time, and using this new parameter to select the file that is generated or not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I probably thought along the lines of testing the code for different file types like pdf and txt.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are suggesting that file_name doesn't help in testing that.I will change as you said.
But should ZT-Client's code test if different file formats work?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this illustrates that various file names should be acceptable to the function. However, how many would we include to show that? Also, file extensions do not necessarily correspond to file types - and I suspect the client/server accepts arbitrary files anyhow? So, if we want to demonstrate that, we could include a fixed string in the test - or as a default value to a test parameter - which looks fairly random.

In case it needs clarifying, using a mock for client.upload means that only ZT code sees this file in any case - it doesn't touch the actual Zulip.Client, and of course not any server.

I mentioned using a path parameter combined with a fixed file_name, since you're not testing eg. ./file_name, path_to/file_name or similar vs just file_name. The file_name could be a constant, with the path varying in the test cases.

{"result": "success", "uri": "http://example.com/success_uri"},
"http://example.com/success_uri",
id="exisiting_file_with_successful_response",
),
case(
"existing_file.txt",
{"result": "failure", "error_message": "Upload failed"},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matches to the code we have (result is not success), but I'm unsure it's the actual form that we would get from the API in the case of an error, which we should be using - at a bare minimum for the value of result. Does it currently match?

None,
id="exisiting_file_with_unsuccessful_response",
),
case(
"non_existing_file.txt",
None,
None,
id="non_exisiting_file_with_no_response",
),
],
)
def test_get_file_upload_uri(
self, mocker, model, file_name, upload_result, expected_result
):
self.client.upload_file = mocker.Mock(return_value=upload_result)
with tempfile.TemporaryDirectory() as temp_dir:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good thinking to use this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle I agree, and using something like this for testing the file existence is certainly a valid approach. One of the downsides is that it involves filesystem actions in each test.

We actually use a similar approach for testing some of run.py right now, using some built-in fixtures that pytest provides. If we take this approach, it would likely be wise to use one of those.

An alternative approach would be to patch open - though that can be tricky since it is built-in - and set it to raise certain exceptions.

if upload_result is not None:
temp_file_path = os.path.join(temp_dir, file_name)
with open(temp_file_path, "w") as temp_file:
temp_file.write("Test content")
else:
temp_file_path = f"random_path/{file_name}"

result = model.get_file_upload_uri(temp_file_path)

if upload_result is not None:
self.client.upload_file.assert_called_once()
else:
self.client.upload_file.assert_not_called()
assert result == expected_result

@pytest.mark.parametrize(
"response, return_value",
[
Expand Down
29 changes: 29 additions & 0 deletions tests/ui_tools/test_boxes.py
Original file line number Diff line number Diff line change
Expand Up @@ -1526,6 +1526,35 @@ def test_keypress_typeahead_mode_autocomplete_key(
else:
self.view.set_footer_text.assert_not_called()

@pytest.mark.parametrize(
"file_name, uri, expected_result",
[
case(
"example.txt",
"http://example.com/example.txt",
"Initial content[example.txt](http://example.com/example.txt)",
id="txt_file",
),
case(
"file.pdf",
"http://example.com/file.pdf",
"Initial content[file.pdf](http://example.com/file.pdf)",
id="pdf_file",
),
],
)
def test_append_uri_and_filename(
self, write_box: WriteBox, file_name: str, uri: str, expected_result: str
) -> None:
initial_edit_text = "Initial content"
write_box.private_box_view(recipient_user_ids=[])
write_box.msg_write_box.edit_text = initial_edit_text

write_box.append_uri_and_filename(file_name, uri)
result_edit_text = write_box.msg_write_box.edit_text
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defining a variable can be avoided here as the value is being used only once.


assert result_edit_text == expected_result
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The behavior of the method is also to set the position of the cursor, so that should be asserted here too.


@pytest.mark.parametrize(
[
"initial_focus_name",
Expand Down
89 changes: 89 additions & 0 deletions tests/ui_tools/test_popups.py
Original file line number Diff line number Diff line change
@@ -1,22 +1,27 @@
from collections import OrderedDict
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional, Tuple
from unittest.mock import MagicMock

import pytest
from pytest import param as case
from pytest_mock import MockerFixture
from urwid import Columns, Pile, Text, Widget

from tests.ui_tools.test_boxes import TestWriteBox
from zulipterminal.api_types import Message
from zulipterminal.config.keys import is_command_key, keys_for_command
from zulipterminal.config.ui_mappings import EDIT_MODE_CAPTIONS
from zulipterminal.helper import CustomProfileData, TidiedUserInfo
from zulipterminal.ui_tools.boxes import WriteBox
from zulipterminal.ui_tools.messages import MessageBox
from zulipterminal.ui_tools.views import (
AboutView,
EditHistoryTag,
EditHistoryView,
EditModeView,
EmojiPickerView,
FileUploadView,
FullRawMsgView,
FullRenderedMsgView,
HelpView,
Expand Down Expand Up @@ -872,6 +877,90 @@ def test_keypress_exit_popup(
assert self.controller.exit_popup.called


class TestFileUploadView:
@pytest.fixture(scope="class")
def write_box(self) -> Any:
return TestWriteBox().write_box
Comment on lines +881 to +883
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to make these tests standalone, rather than coupling them to another set of tests.

Looking below, you pass the detailed write_box from the other test to FileUploadView, but in the test where it's used you've explicitly set it to a simple Mock(). I'd suggest trying that in mock_external_classes instead, and skipping this fixture.


@pytest.fixture(autouse=True)
def mock_external_classes(self, mocker: MockerFixture, write_box: WriteBox) -> None:
self.controller = mocker.Mock()
mocker.patch.object(
self.controller, "maximum_popup_dimensions", return_value=(64, 64)
)
mocker.patch(LISTWALKER, return_value=[])
self.file_upload_view = FileUploadView(
self.controller, write_box, "Upload File"
)

def test_keypress_any_key(
self, widget_size: Callable[[Widget], urwid_Size]
) -> None:
key = "a"
size = widget_size(self.file_upload_view)
self.file_upload_view.keypress(size, key)
assert not self.controller.exit_popup.called

@pytest.mark.parametrize("key", {*keys_for_command("GO_BACK")})
def test_keypress_exit_popup(
self, key: str, widget_size: Callable[[Widget], urwid_Size]
) -> None:
size = widget_size(self.file_upload_view)
self.file_upload_view.keypress(size, key)
assert self.controller.exit_popup.called

@pytest.mark.parametrize(
"file_location, expected_uri, expected_error_message",
[
case(
"example.txt",
"http://example.txt/uploaded_file",
None,
id="txt_file_with_successful_uri",
),
case(
"example.pdf",
"http://example.pdf/uploaded_file",
None,
id="pdf_file_with_successful_uri",
),
case(
"invalid.txt",
"",
["ERROR: Unable to get the URI"],
id="invalid_txt_file_with_error",
),
case(
"invalid.pdf",
"",
["ERROR: Unable to get the URI"],
id="invalid_pdf_file_with_error",
),
],
)
def test__handle_file_upload(
self,
file_location: str,
expected_uri: str,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expected_uri is specifically assigned to something in the setup of the test, so it is not meaningful to think of it as an 'expected' value.

Maybe something like file_upload_response instead? (though this value does not affect the result, only that it is consistently used in the assert).

Note that this is also confusing as it is, since the return type of that function is currently Optional[str] instead of str, and you've not tested for the None values.

expected_error_message: Optional[str],
) -> None:
self.file_upload_view.write_box = MagicMock()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the write_box is set in the external fixtures above, you can (instead/also) assign from there to a shortened variable for the write_box here, which will make the assert statements easier to read.

self.controller.model.get_file_upload_uri.return_value = expected_uri

self.file_upload_view._handle_file_upload(file_location)

self.controller.model.get_file_upload_uri.assert_called_once_with(file_location)
if not expected_error_message:
file_name = Path(file_location).name
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file_name would be good as a test (case) parameter. It may look like a duplicate, but you're processing the result explicitly here instead, and it's clearer as a parameter to the test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. I think you are suggesting this

    def test__handle_file_upload(
        self,
        file_location: str,
        file_name: str,
        expected_uri: str,
        expected_error_message: Optional[str],
    ) -> None:

and directly use the file_name parameter.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, the principle is that ideally a test is as simple as possible - less logic and calculation - and the test cases handle the input and corresponding expected output (expected_* parameters).

self.file_upload_view.write_box.append_uri_and_filename.assert_called_once_with(
file_name, self.file_upload_view.uri
)
else:
self.controller.append_uri_and_filename.assert_not_called()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't exist? It likely still passes due to the perils of Mock without spec.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. will remove it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you do wish to assert on the mocked details, then my point here was more that the object you're asserting on does not exist - and is different to that a few lines above.

self.controller.report_error.assert_called_with(expected_error_message)
self.controller.exit_popup.assert_called()


class TestHelpView:
@pytest.fixture(autouse=True)
def mock_external_classes(self, mocker: MockerFixture) -> None:
Expand Down
5 changes: 5 additions & 0 deletions zulipterminal/config/keys.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,11 @@ class KeyBinding(TypedDict):
'help_text': 'New message to a person or group of people',
'key_category': 'msg_actions',
},
'FILE_UPLOAD': {
'keys': ['ctrl o'],
'help_text': 'Upload file',
'key_category': 'msg_compose',
},
'CYCLE_COMPOSE_FOCUS': {
'keys': ['tab'],
'help_text': 'Cycle through recipient and content boxes',
Expand Down
6 changes: 6 additions & 0 deletions zulipterminal/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,14 @@
from zulipterminal.model import Model
from zulipterminal.platform_code import PLATFORM
from zulipterminal.ui import Screen, View
from zulipterminal.ui_tools.boxes import WriteBox
from zulipterminal.ui_tools.utils import create_msg_box_list
from zulipterminal.ui_tools.views import (
AboutView,
EditHistoryView,
EditModeView,
EmojiPickerView,
FileUploadView,
FullRawMsgView,
FullRenderedMsgView,
HelpView,
Expand Down Expand Up @@ -274,6 +276,10 @@ def show_msg_info(
)
self.show_pop_up(msg_info_view, "area:msg")

def show_file_upload_popup(self, write_box: WriteBox) -> None:
file_upload_view = FileUploadView(self, write_box, "Upload File")
self.show_pop_up(file_upload_view, "area:msg")

def show_emoji_picker(self, message: Message) -> None:
all_emoji_units = [
(emoji_name, emoji["code"], emoji["aliases"])
Expand Down
12 changes: 12 additions & 0 deletions zulipterminal/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

import itertools
import json
import os
import time
from collections import defaultdict
from concurrent.futures import Future, ThreadPoolExecutor, wait
Expand Down Expand Up @@ -560,6 +561,17 @@ def send_stream_message(self, stream: str, topic: str, content: str) -> bool:
notify_if_message_sent_outside_narrow(composition, self.controller)
return message_was_sent

def get_file_upload_uri(self, file_location: str) -> Optional[str]:
if os.path.exists(file_location):
with open(file_location, "rb") as fp:
Comment on lines +565 to +566
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that os.path.exists only checks that this is a valid path, not that it is a file or that it can be accessed by the active user.

There is also the slim possibility that we check it exists and then the file disappears before it is opened. The better pattern is therefore to try to open, and then handle any exceptions relevant to not being able to open the file, or the parent exception class of those we expect, and return None from there.

However, this function now returns None if there is a problem uploading, as well as if the file doesn't exist - the calling function (and user!) cannot distinguish between these cases. In the short term using display_error_if_present for reporting any server error could enable a distinction there - though note that after reporting the error it would still return None, so trigger the same error handling in the caller.

For now we should certainly use a try/except around the open to look for file issues via exception(s). You could experiment with display_error_if_present, but duplicate error messages may look strange.

At a later point it may be clearer to handle only the uploading of an already-opened source file here, and handle other errors where this function is called, making the showing of different errors much simpler - including those arising from different exceptions when opening a proposed upload.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rereading this, it would definitely be better to use the exception approach here, and focus error handling for upload as occurring from the model, compared to error handling with accessing the file from the calling function.

It's possible there could be errors related to processing the upload related to the file handling, but that would be part of the upload process.

result = self.client.upload_file(fp)
if result["result"] == "success":
return result["uri"]
else:
return None
else:
return None

def update_private_message(self, msg_id: int, content: str) -> bool:
request: PrivateMessageUpdateRequest = {
"message_id": msg_id,
Expand Down
11 changes: 11 additions & 0 deletions zulipterminal/ui_tools/boxes.py
Original file line number Diff line number Diff line change
Expand Up @@ -710,6 +710,14 @@ def autocomplete_emojis(

return emoji_typeahead, emojis

def append_uri_and_filename(self, file_name: str, uri: str) -> None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The name of the function suggests that the first argument is the uri and the second the file_name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will change the order of the arguments.

edit_widget = self.contents[self.FOCUS_CONTAINER_MESSAGE][
self.FOCUS_MESSAGE_BOX_BODY
]
edit_widget.edit_text += f"[{file_name}]({str(uri)})"
# Places the cursor after the URI
edit_widget.set_edit_pos(len(edit_widget.get_edit_text()))

def keypress(self, size: urwid_Size, key: str) -> Optional[str]:
if self.is_in_typeahead_mode and not (
is_command_key("AUTOCOMPLETE", key)
Expand All @@ -719,6 +727,9 @@ def keypress(self, size: urwid_Size, key: str) -> Optional[str]:
self.is_in_typeahead_mode = False
self.view.set_footer_text()

if is_command_key("FILE_UPLOAD", key):
self.model.controller.show_file_upload_popup(self)

if is_command_key("SEND_MESSAGE", key):
self.send_stop_typing_status()
if self.compose_box_status == "open_with_stream":
Expand Down
41 changes: 40 additions & 1 deletion zulipterminal/ui_tools/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

import threading
from datetime import datetime
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple, Union

import pytz
Expand Down Expand Up @@ -43,7 +44,7 @@
match_user,
)
from zulipterminal.server_url import near_message_url
from zulipterminal.ui_tools.boxes import PanelSearchBox
from zulipterminal.ui_tools.boxes import PanelSearchBox, WriteBox
from zulipterminal.ui_tools.buttons import (
EmojiButton,
HomeButton,
Expand Down Expand Up @@ -1195,6 +1196,44 @@ def __init__(self, controller: Any, title: str) -> None:
super().__init__(controller, widgets, "HELP", popup_width, title)


class FileUploadView(PopUpView):
def __init__(
self,
controller: Any,
write_box: WriteBox,
title: str,
) -> None:
self.controller = controller
self.model = controller.model
self.write_box = write_box
max_cols, max_rows = controller.maximum_popup_dimensions()
self.predefined_text = urwid.Text("Location : ")
self.file_location_edit = urwid.Edit()
columns = [self.predefined_text, self.file_location_edit]
super().__init__(
controller,
columns,
"GO_BACK",
max_cols,
title,
)

def _handle_file_upload(self, file_location: str) -> None:
self.uri = self.model.get_file_upload_uri(file_location)
if self.uri:
file_path = Path(file_location)
file_name = file_path.name
self.write_box.append_uri_and_filename(file_name, self.uri)
else:
self.controller.report_error(["ERROR: Unable to get the URI"])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error is triggered if the file to upload is the empty string, ie. just pressing enter with no contents in the box. As a user I'd expect that to be equivalent to exiting the box - so it doesn't even try and upload.

That would fit with doing more error handling in this location, as per another comment, though this would be easier to do without changing the way the function is called.

self.controller.exit_popup()

def keypress(self, size: urwid_Size, key: str) -> str:
if is_command_key("ENTER", key):
self._handle_file_upload(self.file_location_edit.edit_text)
return super().keypress(size, key)


class MarkdownHelpView(PopUpView):
def __init__(self, controller: Any, title: str) -> None:
raw_menu_content = [] # to calculate table dimensions
Expand Down