# NAME

HTTP::Upload::FlowJs - handle resumable multi-part HTTP uploads with flowjs

# SYNOPSIS

This synopsis assumes a [Plack](https://metacpan.org/pod/Plack)/[PSGI](https://metacpan.org/pod/PSGI)-like environment. There are
plugins for [Dancer](https://metacpan.org/pod/Dancer) and [Mojolicious](https://metacpan.org/pod/Mojolicious) planned. See
[HTTP::Upload::FlowJs::Examples](https://metacpan.org/pod/HTTP%3A%3AUpload%3A%3AFlowJs%3A%3AExamples) for longer examples.

The `flow.js` workflow assumes that your application handles two kinds of
requests, POST requests for storing the payload data and GET requests for
retrieving information about uploaded parts. You will have to make various calls
to the HTTP::Upload::FlowJs object to validate the incoming request at every
stage.

    use HTTP::Upload::FlowJs;

    my $uploads = '/tmp/flowjs_uploads/';
    my $flowjs = HTTP::Upload::FlowJs->new(
        incomingDirectory => $uploads,
        allowedContentType => sub { $_[0] =~ m!^image/! },
    );

    my @parameter_names = $flowjs->parameter_names();

    # In your POST handler for /upload:
    sub POST_upload {
      my $params = params();

      my %info;
      @info{ @parameter_names } = @{$params}{@parameter_names};
      $info{ localChunkSize } = -s $params{ file };
      # or however you get the size of the uploaded chunk

      # you might want to set this so users don't clobber each others upload
      my $session_id = '';
      my @invalid = $flowjs->validateRequest( 'POST', \%info, $session_id );
      if( @invalid ) {
          warn 'Invalid flow.js upload request:';
          warn $_ for @invalid;
          return [500,[],["Invalid request"]];
          return;
      };

      if( $flowjs->disallowedContentType( \%info, $session_id )) {
          # We can determine the content type, and it's not an image
          return [415,[],["File type disallowed"]];
      };

      my $chunkname = $flowjs->chunkName( \%info, undef );

      # Save or copy the uploaded file
      upload('file')->copy_to($chunkname);

      # Now check if we have received all chunks of the file
      if( $flowjs->uploadComplete( \%info, undef )) {
          # Combine all chunks to final name

          my $digest = Digest::SHA256->new();

          my( $content_type, $ext ) = $flowjs->sniffContentType();
          my $final_name = "file1.$ext";
          open( my $fh, '>', $final_name )
              or die $!;
          binmode $fh;

          my( $ok, @unlink_chunks )
              = $flowjs->combineChunks( \%info, undef, $fh, $digest );
          unlink @unlink_chunks;

          # Notify backend that a file arrived
          print sprintf "File '%s' upload complete\n", $final_name;
      };

      # Signal OK
      return [200,[],[]]
    };

    # This checks whether a file has been received completely or
    # needs to be uploaded again
    sub GET_upload {
      my $params = params();
      my %info;
      @info{ @parameter_names} = @{$params}{@parameter_names};

      my @invalid = $flowjs->validateRequest( 'GET', \%info, session->{connid} );
      if( @invalid ) {
          warn 'Invalid flow.js upload request:';
          warn $_ for @invalid;
          return [500, [], [] ];

      } elsif( $flowjs->disallowedContentType( \%info, $session_id)) {
          # We can determine the content type, and it's not an image
          return [415,[],["File type disallowed"]];

      } else {
          my( $status, @messages )
              = $flowjs->chunkOK( $uploads, \%info, $session_id );
          if( $status != 500 ) {
              # 200 or 416
              return [$status, [], [] ];
          } else {
              warn $_ for @messages;
              return [$status, [], [] ];
          };
      };
    };

# OVERVIEW

[flow.js](https://github.com/flowjs/flow.js) is a client-side Javascript upload
library that uploads
a file in multiple parts. It requires two API points on the server side,
one `GET` API point to check whether a part already has been uploaded
completely and one `POST` API point to send the data of each partial
upload to. This Perl module implements the backend functionality for
both endpoints. It does not implement the handling of the HTTP requests
themselves, but you likely already use a framework like [Mojolicious](https://metacpan.org/pod/Mojolicious)
or [Dancer](https://metacpan.org/pod/Dancer) for that.

# METHODS

## `HTTP::Upload::FlowJs->new`

    my $flowjs = HTTP::Upload::FlowJs->new(
        maxChunkCount => 1000,
        maxFileSize => 10_000_000,
        maxChunkSize => 1024*1024,
        simultaneousUploads => 3,
        allowedContentType => sub {
            my($type) = @_;
            $type =~ m!^image/!; # we only allow for cat images
        },
    );

> **incomingDirectory** - path for the temporary upload parts
>
> Required
>
> **maxChunkCount** - hard maximum chunks allowed for a single upload
>
> Default 1000
>
> **maxFileSize** - hard maximum total file size for a single upload
>
> Default 10\_000\_000
>
> **maxChunkSize** - hard maximum chunk size for a single chunk
>
> Default 1048576
>
> **minChunkSize** - hard minimum chunk size for a single chunk
>
> Default 1024
>
> The minimum chunk size is required since the file type detection
> works on the first chunk. If the first chunk is too small, its file type
> cannot be checked.
>
> **forceChunkSize** - force all chunks to be less or equal than `maxChunkSize`
>
> Default: true
>
> Otherwise, the last chunk will be greater than or equal to `maxChunkSize`
> (the last uploaded chunk will be at least this size and up to two the size).
>
> Note: when `forceChunkSize` is `false` it only make `chunkSize` value in
> ["jsConfig"](#jsconfig) equal to `maxChunkSize/2`.
>
> **simultaneousUploads** - simultaneously allowed uploads per file
>
> Default 3
>
> This is just an indication to the Javascript `flow.js` client
> if you pass it the configuration from this object. This is not enforced
> in any way yet.
>
> **allowedContentType** - subroutine to check the MIME type
>
> The default is to allow any kind of file
>
> If you need more advanced checking, do so after you've determined a file
> upload as complete with `$flowjs->uploadComplete`.
>
> **fileParameterName** - The name of the multipart POST parameter to use for the
> file chunk
>
> Default `file`

More interesting limits would be hard maxima for the number of pending
uploads or the number of outstanding chunks per user/session. Checking
these would entail a call to `glob` for each check and thus would be
fairly disk-intensive on some systems.

## `$flowjs->incomingDirectory`

Return the incoming directory name.

## `$flowjs->mime`

Return the [MIME::Detect](https://metacpan.org/pod/MIME%3A%3ADetect) instance.

## `$flowjs->jsConfig`

## `$flowjs->jsConfigStr`

    # Perl HASH
    my $config = $flowjs->jsConfig(
        target => '/upload',
    );

    # JSON string
    my $config = $flowjs->jsConfigStr(
        target => '/upload',
    );

Create a JSON string that encapsulates the configuration of the Perl
object for inclusion with the JS side of the world.

## `$flowjs->parameter_names`

    my $params = params();                 # request params
    my @parameter_names = $flowjs->parameter_names; # params needed by Flowjs

    my %info;
    @info{ @parameter_names } = @{$params}{@parameter_names};

    $info{ file }           = $params{ file };
    $info{ localChunkSize } = -s $params{ file };

    my @invalid = $flowjs->validateRequest( 'POST', \%info );

Returns needed params for validating request.

## `$flowjs->validateRequest`

    my $session_id = '';
    my @invalid = $flowjs->validateRequest( 'POST', \%info, $session_id );
    if( @invalid ) {
        warning 'Invalid flow.js upload request:';
        warning $_ for @invalid;
        status 500;
        return;
    };

Does formal validation of the request HTTP parameters. It does not
check previously stored information.

**Note** when `POST` there are addition required params `localChunkSize`
and `$self-`{fileParameterName}> (default 'file').

## `$flowJs->expectedChunkSize`

    my $expectedSize = $flowJs->expectedChunkSize( $info, $chunkIndex );

Returns the file size we expect for the chunk `$chunkIndex`. The index
starts at 1, if it is not passed in or zero, we assume it is for the current
chunk as indicated by `$info`.

## `$flowjs->resetUploadDirectories`

    if( $firstrun or $wipe ) {
        $flowJs->resetUploadDirectories( $wipe )
    };

Creates the directory for incoming uploads. If `$wipe`
is passed, it will remove all partial files from the directory.

## `$flowjs->chunkName`

    my $target = $flowjs->chunkName( $info, $sessionid );

Returns the local filename of the chunk described by `$info` and
the `$sessionid` if given. An optional index can be passed in as
the third parameter to get the filename of another chunk than
the current chunk.

    my $target = $flowjs->chunkName( $info, $sessionid, 1 );
    # First chunk

## `$flowjs->chunkOK`

    my( $status, @messages ) = $flowjs->chunkOK( $info, $sessionPrefix );
    if( $status == 500 ) {
        warn $_ for @messages;
        return [ 500, [], [] ]

    } elsif( $status == 200 ) {
        # That chunk exists and has the right size
        return [ 200, [], [] ]

    } else {
        # That chunk does not exist and should be uploaded
        return [ 416, [],[] ]
    }

## `$flowjs->uploadComplete( $info, $sessionPrefix=undef )`

    if( $flowjs->uploadComplete($info, $sessionPrefix) ) {
        # do something with the chunks
    }

## `$flowjs->chunkFh`

    my $fh = $flowjs->chunkFh( $info, $sessionid, $index );

Returns an opened filehandle to the chunk described by `$info`. The session
and the index are optional.

## `$flowjs->chunkContent`

    my $content = $flowjs->chunkContent( $info, $sessionid, $index );

Returns the content of a chunk described by `$info`. The session
and the index are optional.

## `$flowjs->disallowedContentType( $info, $session )`

    if( $flowjs->disallowedContentType( $info, $session )) {
        return 415, "This type of file is not allowed";
    };

Checks that the subroutine validator passed in the constructor allows
this MIME type. Unrecognized files will be blocked.

## `$flowjs->sniffContentType( $info, $session )`

    my( $content_type, $image_ext ) = $flowjs->sniffContentType( $info, $session );
    if( !defined $content_type ) {
        # we need more chunks uploaded to check the content type

    } elsif( $content_type eq '' ) {
        # We couldn't determine what the content type is?!
        return 415, "This type of upload is not allowed";

    } elsif( $content_type !~ m!^image/(jpeg|png|gif)$!i ) {
        return 415, "This type of upload is not allowed";

    } else {
        # We allow this upload to continue, as it seems to have
        # an appropriate content type
    };

This allows for finer-grained checking of the MIME-type. See also
the `allowedContentType` argument in the constructor and
[->disallowedContentType](https://metacpan.org/pod/-%3EdisallowedContentType) for a more convenient way to quickly
check the upload type.

## `$flowjs->combineChunks $info, $sessionPrefix, $target_fh, $digest )`

    if( not $flowjs->uploadComplete($info, $sessionPrefix) ) {
        print "Upload not yet completed\n";
        return;
    };

    open my $file, '>', 'user_upload.jpg'
        or die "Couldn't create final file 'user_upload.jpg': $!";
    binmode $file;
    my $digest = Digest::SHA256->new();
    my($ok,@unlink) = $flowjs->combineChunks( $info, undef, $file, $digest );
    close $file;

    if( $ok ) {
        print "Received upload OK, removing temporary upload files\n";
        unlink @unlink;
        print "Checksum: " . $digest->md5hex;
    } else {
        # whoops
        print "Received upload failed, removing target file\n";
        unlink 'user_upload.jpg';
    };

## `$flowjs->pendingUploads`

    my $uploading = $flowjs->pendingUploads();

In scalar context, returns the number of pending uploads. In list context,
returns the list of filenames that belong to the pending uploads. This list
can be larger than the number of pending uploads, as one upload can have more
than one chunk.

## `$flowjs->staleUploads( $timeout, $now )`

    my @stale_files = $flowjs->staleUploads(3600);

In scalar context, returns the number of stale uploads. In list context,
returns the list of filenames that belong to the stale uploads.

An upload is considered stale if no part of it has been written to since
`$timeout` seconds ago.

The optional `$timeout` parameter is the minimum age of an incomplete upload
before it is considered stale.

The optional `$now` parameter is the point of reference for `$timeout`.
It defaults to `time`.

## `$flowjs->purgeStaleOrInvalid( $timeout, $now )`

    my @errors = $flowjs->purgeStaleOrInvalid();

Routine to delete all stale uploads and uploads with an invalid
file type.

This is mostly a helper routine to run from a cron job.

Note that if you allow uploads of multiple flowJs instances into the same
directory, they need to all have the same allowed file types or this method
will delete files from another instance.

# REPOSITORY

The public repository of this module is
[https://github.com/Corion/HTTP-Upload-FlowJs](https://github.com/Corion/HTTP-Upload-FlowJs).

# SUPPORT

The public support forum of this module is
[https://perlmonks.org/](https://perlmonks.org/).

# BUG TRACKER

Please report bugs in this module via the RT CPAN bug queue at
[https://rt.cpan.org/Public/Dist/Display.html?Name=HTTP-Upload-FlowJs](https://rt.cpan.org/Public/Dist/Display.html?Name=HTTP-Upload-FlowJs)
or via mail to [bug-http-upload-flowjs@rt.cpan.org](https://metacpan.org/pod/bug-http-upload-flowjs%40rt.cpan.org).

# AUTHOR

Max Maischein `corion@cpan.org`

# COPYRIGHT (c)

Copyright 2009-2018 by Max Maischein `corion@cpan.org`.

# LICENSE

This module is released under the same terms as Perl itself.
