How To Upload Large CSV File Using Queue In Laravel 9

Websolutionstuff | Sep-16-2022 | Categories : Laravel

In this article, we will see how to upload a large CSV file using queue in laravel 9. Here we will learn large numbers of records inserted into the database using the queue job in laravel. Also, you can learn how to create queue jobs in laravel 9.

We can upload more than 50K+ records to our database. So, it takes more loading time to insert into the database. For that reason, we will use the laravel queue job. If we use queue job all processes can background in our application and it can improve or boost the performance.

So, let's see upload a large CSV file in the database using laravel 9, and import a large CSV file using the queue in laravel 9.

Step 1: Install Laravel 9

In this step, we will create a laravel 9 application using the following command.

composer create-project --prefer-dist laravel/laravel laravel-9-app

 

 

 Step 2: Create Migration and Model

In this step, we will create migration and model using the following command.

php artisan make:model Product -m

Add the below code in database/migrations/create_products_table.php:

public function up()
{
    Schema::create('products', function (Blueprint $table) {
        $table->bigIncrements('id');
        $table->string('name');
        $table->string('sku');
        $table->string('price');
        $table->timestamps();
    });
}

Now, run the migration using the below command.

php artisan migrate

 Changes in the app/Models/Product.php file.

<?php
 
namespace App\Models;
 
use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
 
class Product extends Model
{
    use HasFactory;
    protected $guarded = [];
}

 

 

Step 3: Create Route

Now, we will create two routes for the file upload form, and then we have to upload it to the server.

routes/web.php

use Illuminate\Support\Facades\Route;
use App\Http\Controllers\ProductController;

/*
|--------------------------------------------------------------------------
| Web Routes
|--------------------------------------------------------------------------
|
| Here is where you can register web routes for your application. These
| routes are loaded by the RouteServiceProvider within a group which
| contains the "web" middleware group. Now create something great!
|
*/

Route::get('/', [ProductController::class,'index'])->name('upload');
Route::post('/', [ProductController::class,'upload_csv_file']);

 

Step 4: Create Controller

In this step, we will create a controller.

App\Http\Controllers\SaleController.php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use App\Models\Product;
use App\Jobs\ProductCSVJob;

class ProductController extends Controller
{
    public function index()
    {
        return view('welcome');
    }

    public function upload_csv_file(Request $request)
    {
        if( $request->has('csv') ) {

            $csv    = file($request->csv);
            $chunks = array_chunk($csv,1000);
            $header = [];

            foreach ($chunks as $key => $chunk) {
            $data = array_map('str_getcsv', $chunk);
                if($key == 0){
                    $header = $data[0];
                    unset($data[0]);
                }

                ProductCSVJob::dispatch($data, $header);                
            }

        }
        return "please upload csv file";
    }
}

 

 

Step 5: Create a View

In this step, we will create a blade view to show our file upload form. So, create it to complete the laravel CSV file upload example with queue job.

resources/views/welcome.blade.php


<!DOCTYPE html>
<html lang="{{ str_replace('_', '-', app()->getLocale()) }}">
    <head>
        <meta charset="utf-8">
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <title>How To Upload Large CSV File Using Queue In Laravel 9 - Websolutionstuff</title>
    </head>
    <body class="antialiased">
        <div class="relative flex items-top justify-center min-h-screen bg-gray-100 dark:bg-gray-900 sm:items-center py-4 sm:pt-0">

            <form action="{{ route('upload') }}" method="POST" enctype="multipart/form-data">
                @csrf
                <input type="file" name="csv">
                <input type="submit" value="submit">
          </form>

        </div>
    </body>
</html>

 

Step 6: Create a Job

In this step, we will create a job to process our large CSV file. So, run the following command.

php artisan make:job ProductCSVJob

Now, update the ProductCSVJob code like below.

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use App\Models\Product;

class ProductCSVJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    
    public $header;
    public $data;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct($data, $header)
    {
        $this->data = $data;
        $this->header = $header;
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        foreach ($this->data as $product) {
            $product_csv_data = array_combine($this->header,$product);
            Product::create($product_csv_data);
        }
    }
}

 

 

 Step 7: Create Queue Table

Now, we will create a 'jobs' table in the database. So, copy the below command and run it in your terminal.

php artisan queue:table

php artisan migrate​

 After uploading CSV File run the below command in your terminal to upload CSV files into the database.

php artisan queue:listen

 


You might also like :

Recommended Post
Featured Post
Laravel: 10 Steps to Becoming a Laravel Expert
Laravel: 10 Steps to Becoming...

In this article, we will see 10 steps to becoming a laravel expert. Here, we will learn about how to become a larav...

Read More

May-26-2023

Laravel 9 CRUD Operation Example
Laravel 9 CRUD Operation Examp...

As we know Laravel 9 was officially released yesterday with many new functionalities like Symfony 6.0 components, Symfon...

Read More

Feb-10-2022

How To Get Last 30 Days Record In Laravel 8
How To Get Last 30 Days Record...

in this tutorial, we see how to get last 30 days record in laravel 8. You can simply get the last 30 days reco...

Read More

Feb-02-2022

Generate Dynamic Sitemap in Laravel
Generate Dynamic Sitemap in La...

Here we will see how to generate dynamic sitemap in laravel. As we know sitemap is very important part of seo, sitemap i...

Read More

Jul-05-2021