[WIP] Added #5957 - Flysystem support (#6262)

* Added AWS url to example env

* Upgrader - added check for new storage path and attempt to move

* Ignore symlink

* Updated paths for models

* Moved copy methods

* Added AWS_URL support

For some reasin, Flysystem was generating the wrong AWS url (with a region included)

* Switch to Flysystem for image uploads

* Nicer display of image preview

* Updated image preview on edit blades to use Flysystem

* Twiddled some more paths

* Working filesystems config

* Updated Asset Models and Departments to use Flysystem

* Janky workaround for differing S3/local urls/paths

* Try to smartly use S3 as public disk if S3 is configured

* Use public disk Storage options for public files

* Additional transformer edits for Flysystem

* Removed debugging

* Added missing use Storage directive

* Updated seeders to use Flysystem

* Default logo

* Set a default width

We can potentially override this in settings later

* Use Flysystem for logo upload

* Update downloadFile to use Flysystem

* Updated AssetFilesController to use Flysystem

* Updated acceptance signatures to use Flysystem

* Updated signature view to use Flysystem

This isn’t working 100% yet

* Use Flysystem facade for displaying asset image

* Set assets path

Should clean all these up when we’re done here

* Added Rackspace support for Flysystem

* Added Flysystem migrator console command

* Added use Storage directive for categories

* Added user avatars to Flysystem

* Added profile avatar to Flysystem

* Added the option to delete local files with the migrator

* Added a check to prevent people from trying to move from local to local

* Fixed the selectlists for Flysystem

* Fixed the getImageUrl method to reflect Flysystem

* Fixed AWS copy process

* Fixed models path

* More selectlist updates for Flysystem

* Updated example .envs with updated env variable names

* *sigh*

* Updated non-asset getImageUrl() methods to use Flysystem

* Removed S3 hardcoding

* Use Flysystem in email headers

* Fixed typo

* Removed camera support from asset file upload

We’ll find a way to add this in later (and add that support to all of the other image uploads as well)

* Fixed path for categories

* WIP - Switched to standard handleImages for asset upload.

This is currently broken as I refact the handleImages method. Because the assets store/create methods use their own Form Request, the handleImages method doesn’t exist in that Form Request so it wil error now.

* Fixed css URL error

* Updated Debugbar to latest version (#6265)

v3.2 adds support for Laravel 5.7

* Fixed: Missing CSS file in basic.blade.php (#6264)

* Fixed missing CSS file in basic.blade.php

* Added

* Changed stylesheet import for authorize.blade.php

* Updated composer lock

* Added AWS_BUCKET_ROOT as env variable

* Use nicer image preview for logo upload

* Removed AssetRequest form request

* Removed asset form request, moved custom field validation into model

* Added additional help text for logo upload

* Increased the size of the image resize - should make this a setting tho

* Few more formatting tweaks to logo section of branding blade preview

* Use Flysystem for asset/license file uploads

* Use Flysystem for removing images from models that have been deleted

* Enable backups to use Flysystem

This only handles part of the problem. This just makes it so we can ship files to S3 if we want, but does not account for how we backup files that are hosted on S3

* Use Flysystem to download license files

* Updated audits to use Flysystem

.env.example

@@ -85,10 +85,12 @@ REDIS_PORT-null
 # --------------------------------------------
 # OPTIONAL: AWS S3 SETTINGS
 # --------------------------------------------
-AWS_SECRET=null
-AWS_KEY=null
-AWS_REGION=null
+AWS_SECRET_ACCESS_KEY=null
+AWS_ACCESS_KEY_ID=null
+AWS_DEFAULT_REGION=null
 AWS_BUCKET=null
+AWS_BUCKET_ROOT=null
+AWS_URL=null
 
 # --------------------------------------------
 # OPTIONAL: LOGIN THROTTLING

.env.testing

@@ -40,10 +40,12 @@ IMAGE_LIB=gd
 # --------------------------------------------
 # OPTIONAL: AWS S3 SETTINGS
 # --------------------------------------------
-AWS_SECRET=null
-AWS_KEY=null
-AWS_REGION=null
+AWS_SECRET_ACCESS_KEY=null
+AWS_ACCESS_KEY_ID=null
+AWS_DEFAULT_REGION=null
 AWS_BUCKET=null
+AWS_BUCKET_ROOT=null
+AWS_URL=null
 
 
 # --------------------------------------------

.env.testing-ci

@@ -40,10 +40,12 @@ IMAGE_LIB=gd
 # --------------------------------------------
 # OPTIONAL: AWS S3 SETTINGS
 # --------------------------------------------
-AWS_SECRET=null
-AWS_KEY=null
-AWS_REGION=null
+AWS_SECRET_ACCESS_KEY=null
+AWS_ACCESS_KEY_ID=null
+AWS_DEFAULT_REGION=null
 AWS_BUCKET=null
+AWS_BUCKET_ROOT=null
+AWS_URL=null
 
 
 # --------------------------------------------

app/Console/Commands/MoveUploadsToNewDisk.php

@@ -0,0 +1,183 @@
+<?php
+
+namespace App\Console\Commands;
+
+
+use Illuminate\Console\Command;
+use Illuminate\Support\Facades\Storage;
+
+class MoveUploadsToNewDisk extends Command
+{
+    /**
+     * The name and signature of the console command.
+     *
+     * @var string
+     */
+    protected $signature = 'snipeit:move-uploads {delete_local?}';
+
+    /**
+     * The console command description.
+     *
+     * @var string
+     */
+    protected $description = 'This will move your uploaded files to whatever your current disk is.';
+
+    /**
+     * Create a new command instance.
+     *
+     * @return void
+     */
+    public function __construct()
+    {
+        parent::__construct();
+    }
+
+    /**
+     * Execute the console command.
+     *
+     * @return mixed
+     */
+    public function handle()
+    {
+
+        if (config('filesystems.default')=='local') {
+            $this->error('Your current disk is set to local so we cannot proceed.');
+            $this->warn("Please configure your .env settings for S3 or Rackspace, \nand change your FILESYSTEM_DISK value to 's3' or 'rackspace'.");
+            return false;
+        }
+        $delete_local = $this->argument('delete_local');
+
+        $public_uploads['accessories'] = glob('storage/app/public/accessories'."/*.*");
+        $public_uploads['assets'] = glob('storage/app/public/assets'."/*.*");
+        $public_uploads['avatars'] = glob('storage/app/public/avatars'."/*.*");
+        $public_uploads['barcodes'] = glob('storage/app/public/barcodes'."/*.*");
+        $public_uploads['categories'] = glob('storage/app/public/categories'."/*.*");
+        $public_uploads['companies'] = glob('storage/app/public/companies'."/*.*");
+        $public_uploads['components'] = glob('storage/app/public/components'."/*.*");
+        $public_uploads['consumables'] = glob('storage/app/public/consumables'."/*.*");
+        $public_uploads['departments'] = glob('storage/app/public/departments'."/*.*");
+        $public_uploads['locations'] = glob('storage/app/public/locations'."/*.*");
+        $public_uploads['manufacturers'] = glob('storage/app/public/manufacturers'."/*.*");
+        $public_uploads['suppliers'] = glob('storage/app/public/suppliers'."/*.*");
+        $public_uploads['assetmodels'] = glob('storage/app/public/models'."/*.*");
+
+
+        // iterate files
+        foreach($public_uploads as $public_type => $public_upload)
+        {
+            $type_count = 0;
+            $this->info("\nThere are ".count($public_upload).' PUBLIC '.$public_type.' files.');
+
+            for ($i = 0; $i < count($public_upload); $i++) {
+                $type_count++;
+                $filename = basename($public_upload[$i]);
+
+                try  {
+                    Storage::disk('public')->put($public_type.'/'.$filename, file_get_contents($public_upload[$i]));
+                    $new_url = Storage::disk('public')->url($public_type.'/'.$filename, $filename);
+                    $this->info($type_count.'. PUBLIC: '.$filename.' was copied to '.$new_url);
+                } catch (\Exception $e) {
+                    \Log::debug($e);
+                    $this->error($e);
+                }
+
+            }
+
+        }
+
+        $logos = glob('public/uploads'."/logo*.*");
+        $this->info("\nThere are ".count($logos).' files that might be logos.');
+        $type_count=0;
+
+        for ($l = 0; $l < count($logos); $l++) {
+            $type_count++;
+            $filename = basename($logos[$l]);
+            $new_url = Storage::disk('public')->url($logos[$l], file_get_contents($public_upload[$i]));
+            $this->info($type_count.'. LOGO: '.$filename.' was copied to '.$new_url);
+        }
+
+        $private_uploads['assets'] = glob('storage/private_uploads/assets'."/*.*");
+        $private_uploads['signatures'] = glob('storage/private_uploads/signatures'."/*.*");
+        $private_uploads['audits'] = glob('storage/private_uploads/audits'."/*.*");
+        $private_uploads['assetmodels'] = glob('storage/private_uploads/assetmodels'."/*.*");
+        $private_uploads['imports'] = glob('storage/private_uploads/imports'."/*.*");
+        $private_uploads['licenses'] = glob('storage/private_uploads/licenses'."/*.*");
+        $private_uploads['users'] = glob('storage/private_uploads/users'."/*.*");
+
+
+        foreach($private_uploads as $private_type => $private_upload)
+        {
+            $this->info("\nThere are ".count($private_upload).' PRIVATE '.$private_type.' files.');
+            // $this->info(print_r($private_upload, true));
+
+            $type_count = 0;
+            for ($x = 0; $x < count($private_upload); $x++) {
+                $type_count++;
+                $filename = basename($private_upload[$x]);
+
+                try  {
+                    Storage::disk('private_uploads')->put($private_type.'/'.$filename, file_get_contents($public_upload[$i]));
+                    $new_url = Storage::url($private_type.'/'.$filename, $filename);
+                    $this->info($type_count.'. PRIVATE: '.$filename.' was copied to '.$new_url);
+
+                } catch (\Exception $e) {
+                    \Log::debug($e);
+                    $this->error($e);
+                }
+
+            }
+
+        }
+
+
+        if ($delete_local=='true') {
+            $public_delete_count = 0;
+            $private_delete_count = 0;
+
+            $this->info("\n\n");
+            $this->error('!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
+            $this->warn("\nTHIS WILL DELETE ALL OF YOUR LOCAL UPLOADED FILES. \n\nThis cannot be undone, so you should take a backup of your system before you proceed.\n");
+            $this->error('!!!!!!!!!!!!!!!!!!!!!!!!!!!!! WARNING!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!');
+
+            if ($this->confirm("Do you wish to continue?")) {
+
+                foreach($public_uploads as $public_type => $public_upload) {
+
+                    for ($i = 0; $i < count($public_upload); $i++) {
+                        $filename = $public_upload[$i];
+                        try {
+                            unlink($filename);
+                            $public_delete_count++;
+                        } catch (\Exception $e) {
+                            \Log::debug($e);
+                            $this->error($e);
+                        }
+
+                    }
+                }
+
+                foreach($private_uploads as $private_type => $private_upload)
+                {
+
+                    for ($i = 0; $i < count($private_upload); $i++) {
+                        $filename = $private_upload[$i];
+                        try {
+                            unlink($filename);
+                            $private_delete_count++;
+                        } catch (\Exception $e) {
+                            \Log::debug($e);
+                            $this->error($e);
+                        }
+
+                    }
+                }
+
+                $this->info($public_delete_count." PUBLIC local files and ".$private_delete_count." PRIVATE local files were delete from your filesystem.");
+            }
+        }
+
+
+
+
+    }
+}

app/Console/Kernel.php

@@ -32,6 +32,7 @@ class Kernel extends ConsoleKernel
         Commands\SyncAssetCounters::class,
         Commands\RestoreDeletedUsers::class,
         Commands\SendCurrentInventoryToUsers::class,
+        Commands\MoveUploadsToNewDisk::class,
     ];
 
     /**

app/Http/Controllers/Accessories/AccessoriesController.php

@@ -9,6 +9,7 @@
 use Illuminate\Support\Facades\Auth;
 use Illuminate\Http\Request;
 use Redirect;
+use Illuminate\Support\Facades\Storage;
 
 /** This controller handles all actions related to Accessories for
  * the Snipe-IT Asset Management application.
@@ -170,6 +171,15 @@ public function destroy($accessoryId)
         if ($accessory->hasUsers() > 0) {
              return redirect()->route('accessories.index')->with('error', trans('admin/accessories/message.assoc_users', array('count'=> $accessory->hasUsers())));
         }
+
+        if ($accessory->image) {
+            try  {
+                Storage::disk('public')->delete('accessories'.'/'.$accessory->image);
+            } catch (\Exception $e) {
+                \Log::debug($e);
+            }
+        }
+
         $accessory->delete();
         return redirect()->route('accessories.index')->with('success', trans('admin/accessories/message.delete.success'));
     }

app/Http/Controllers/Account/AcceptanceController.php

@@ -15,6 +15,7 @@
 use Illuminate\Http\Request;
 use Illuminate\Support\Facades\Auth;
 use Illuminate\Support\Str;
+use Illuminate\Support\Facades\Storage;
 
 class AcceptanceController extends Controller {
 
@@ -40,7 +41,7 @@ public function create($id) {
         $acceptance = CheckoutAcceptance::find($id);
 
         if (is_null($acceptance)) {
-            return redirect()->reoute('account.accept')->with('error', trans('admin/hardware/message.does_not_exist'));
+            return redirect()->route('account.accept')->with('error', trans('admin/hardware/message.does_not_exist'));
         }
 
         if (! $acceptance->isPending()) {
@@ -70,7 +71,7 @@ public function store(Request $request, $id) {
         $acceptance = CheckoutAcceptance::find($id);
 
         if (is_null($acceptance)) {
-            return redirect()->reoute('account.accept')->with('error', trans('admin/hardware/message.does_not_exist'));
+            return redirect()->route('account.accept')->with('error', trans('admin/hardware/message.does_not_exist'));
         }
 
         if (! $acceptance->isPending()) {
@@ -92,13 +93,17 @@ public function store(Request $request, $id) {
         /**
          * Get the signature and save it
          */
+
+        if (!Storage::exists('private_uploads/signatures'))  Storage::makeDirectory('private_uploads/signatures', 775);
+
+
+
         if ($request->filled('signature_output')) {
-            $path = config('app.private_uploads').'/signatures';
             $sig_filename = "siglog-" .Str::uuid() . '-'.date('Y-m-d-his').".png";
             $data_uri = e($request->input('signature_output'));
             $encoded_image = explode(",", $data_uri);
             $decoded_image = base64_decode($encoded_image[1]);
-            file_put_contents($path."/".$sig_filename, $decoded_image);
+            Storage::put('private_uploads/signatures/'.$sig_filename, (string)$decoded_image);
         }
 
 
@@ -122,4 +127,4 @@ public function store(Request $request, $id) {
 
         return redirect()->to('account/accept')->with('success', $return_msg);
     }	
-}
+}

app/Http/Controllers/Api/AssetModelsController.php

@@ -9,6 +9,7 @@
 use App\Http\Transformers\AssetModelsTransformer;
 use App\Http\Transformers\AssetsTransformer;
 use App\Http\Transformers\SelectlistTransformer;
+use Illuminate\Support\Facades\Storage;
 
 
 /**
@@ -177,7 +178,7 @@ public function destroy($id)
 
         if ($assetmodel->image) {
             try  {
-                unlink(public_path().'/uploads/models/'.$assetmodel->image);
+                Storage::disk('public')->delete('assetmodels/'.$assetmodel->image);
             } catch (\Exception $e) {
                 \Log::error($e);
             }
@@ -234,7 +235,7 @@ public function selectlist(Request $request)
                 $assetmodel->use_text .=  ' (#'.e($assetmodel->model_number).')';
             }
 
-            $assetmodel->use_image = ($settings->modellistCheckedValue('image') && ($assetmodel->image)) ? url('/').'/uploads/models/'.$assetmodel->image : null;
+            $assetmodel->use_image = ($settings->modellistCheckedValue('image') && ($assetmodel->image)) ? Storage::disk('public')->url('assetmodels/'.e($assetmodel->image)) : null;
         }
 
         return (new SelectlistTransformer)->transformSelectlist($assetmodels);

app/Http/Controllers/Api/AssetsController.php

@@ -3,7 +3,6 @@
 
 use App\Helpers\Helper;
 use App\Http\Controllers\Controller;
-use App\Http\Requests\AssetRequest;
 use App\Http\Requests\AssetCheckoutRequest;
 use App\Http\Transformers\AssetsTransformer;
 use App\Models\Asset;
@@ -400,7 +399,7 @@ public function selectlist(Request $request)
      * @since [v4.0]
      * @return JsonResponse
      */
-    public function store(AssetRequest $request)
+    public function store(Request $request)
     {
 
         $this->authorize('create', Asset::class);
@@ -431,7 +430,7 @@ public function store(AssetRequest $request)
         // Update custom fields in the database.
         // Validation for these fields is handled through the AssetRequest form request
         $model = AssetModel::find($request->get('model_id'));
-        if ($model->fieldset) {
+        if (($model) && ($model->fieldset)) {
             foreach ($model->fieldset->fields as $field) {
                 $asset->{$field->convertUnicodeDbSlug()} = e($request->input($field->convertUnicodeDbSlug(), null));
             }

app/Http/Controllers/Api/CategoriesController.php

@@ -8,6 +8,7 @@
 use App\Models\Category;
 use App\Http\Transformers\CategoriesTransformer;
 use App\Http\Transformers\SelectlistTransformer;
+use Illuminate\Support\Facades\Storage;
 
 class CategoriesController extends Controller
 {
@@ -158,7 +159,7 @@ public function selectlist(Request $request, $category_type = 'asset')
         // This lets us have more flexibility in special cases like assets, where
         // they may not have a ->name value but we want to display something anyway
         foreach ($categories as $category) {
-            $category->use_image = ($category->image) ? url('/').'/uploads/categories/'.$category->image : null;
+            $category->use_image = ($category->image) ? Storage::disk('public')->url('categories/'.$category->image, $category->image) : null;
         }
 
         return (new SelectlistTransformer)->transformSelectlist($categories);

app/Http/Controllers/Api/CompaniesController.php

@@ -178,7 +178,7 @@ public function selectlist(Request $request)
         // This lets us have more flexibility in special cases like assets, where
         // they may not have a ->name value but we want to display something anyway
         foreach ($companies as $company) {
-            $company->use_image = ($company->image) ? url('/').'/uploads/companies/'.$company->image : null;
+            $company->use_image = ($company->image) ? Storage::disk('public')->url('companies/'.$company->image, $company->image) : null;
         }
 
         return (new SelectlistTransformer)->transformSelectlist($companies);