Storing Uploaded Files and Serving Them in Express

"Your file has been uploaded... but where did it actually go?"

Every developer has been there. You implement file upload, the response says "success," but now you have no idea where the file lives or how to access it.

File uploads aren't magic. They're bytes traveling from a user's computer to your server's hard drive. Let me show you exactly what happens, where files go, and most importantly - how to serve them back safely.

Where Uploaded Files Are Stored

When a file is uploaded, it doesn't automatically save anywhere. You explicitly tell multer (or your upload middleware) where to put it.

Memory Storage vs Disk Storage

Multer offers two storage engines:

Storage Type	Where File Lives	Best For
MemoryStorage	RAM (as Buffer object)	Small files, processing before saving, cloud uploads
DiskStorage	Hard drive (permanent file)	Permanent storage, direct serving, large files

Memory Storage Example

const multer = require('multer');
const storage = multer.memoryStorage();
const upload = multer({ storage: storage });

app.post('/upload', upload.single('file'), (req, res) => {
  // File lives in RAM as a Buffer
  console.log(req.file.buffer); // <Buffer 89 50 4e 47 0d 0a ...>
  console.log(req.file.size); // File size in bytes
  
  // Nothing is saved to disk unless you explicitly write it
  // Great for: resize image, then upload to S3
});

Key insight: With memory storage, the file vanishes when the request ends unless you save it somewhere.

Disk Storage Example (Most Common)

const multer = require('multer');
const path = require('path');
const fs = require('fs');

// Ensure upload directory exists
const uploadDir = './uploads';
if (!fs.existsSync(uploadDir)) {
  fs.mkdirSync(uploadDir, { recursive: true });
}

const storage = multer.diskStorage({
  destination: function (req, file, cb) {
    // Where files are saved
    cb(null, './uploads');
  },
  filename: function (req, file, cb) {
    // What the file is named
    const uniqueSuffix = Date.now() + '-' + Math.round(Math.random() * 1E9);
    cb(null, uniqueSuffix + path.extname(file.originalname));
  }
});

const upload = multer({ storage: storage });

app.post('/upload', upload.single('file'), (req, res) => {
  // File is now physically saved at: ./uploads/1678901234567-123456789.jpg
  console.log('File saved at:', req.file.path);
  
  res.json({
    filename: req.file.filename,
    path: req.file.path,
    size: req.file.size
  });
});

File Storage Folder Structure

A well-organized upload directory is essential for maintainability.

Recommended Folder Structure

Creating This Structure Programmatically

const fs = require('fs');
const path = require('path');

const directories = [
  './uploads/avatars',
  './uploads/documents/2024/Q1',
  './uploads/documents/2024/Q2',
  './uploads/images/products',
  './uploads/images/blog',
  './uploads/temp'
];

directories.forEach(dir => {
  fs.mkdirSync(dir, { recursive: true });
});

Dynamic File Organization by User

First, let's organize uploads by user - even temporarily before they go to the cloud.

javascript const multer = require('multer'); const path = require('path'); const fs = require('fs');

// Dynamic storage organized by user const createUserStorage = (req, file, cb) => { const userId = req.user?.id || 'anonymous'; const timestamp = Date.now(); const date = new Date(); const yearMonth = \({date.getFullYear()}-\){String(date.getMonth() + 1).padStart(2, '0')};

// Temporary storage path: uploads/temp/userId/year-month/ const tempDir = path.join(__dirname, 'uploads', 'temp', userId, yearMonth);

if (!fs.existsSync(tempDir)) { fs.mkdirSync(tempDir, { recursive: true }); }

cb(null, tempDir); };

const generateSecureFilename = (originalname) => { const timestamp = Date.now(); const random = Math.random().toString(36).substring(2, 10); const ext = path.extname(originalname); const sanitized = originalname .replace(/[^a-zA-Z0-9.-]/g, '-') .substring(0, 40);

return \({timestamp}-\){random}-\({sanitized}\){ext}; };

const storage = multer.diskStorage({ destination: createUserStorage, filename: (req, file, cb) => { cb(null, generateSecureFilename(file.originalname)); } });

const upload = multer({ storage: storage, limits: { fileSize: 10 * 1024 * 1024 } // 10MB limit });

Complete Pipeline: Upload → Validate → ImageKit → Database → Cleanup

Here's the entire flow with ImageKit validation integrated.

Step 1: Install Dependencies

bash npm install multer imagekit mongoose dotenv Step 2: Configure Environment Variables env

.env

IMAGEKIT_PUBLIC_KEY=your_public_key IMAGEKIT_PRIVATE_KEY=your_private_key IMAGEKIT_URL_ENDPOINT=https://ik.imagekit.io/your\_app\_id

MONGODB_URI=mongodb://localhost:27017/file_uploads

UPLOAD_TEMP_DIR=./uploads/temp MAX_FILE_SIZE=10485760

Step 3: Database Models

// models/File.js
const mongoose = require('mongoose');

const fileSchema = new mongoose.Schema({
  // User association
  userId: {
    type: mongoose.Schema.Types.ObjectId,
    ref: 'User',
    required: true,
    index: true
  },
  
  // Original file metadata
  originalName: {
    type: String,
    required: true
  },
  
  // ImageKit specific fields
  fileId: {
    type: String,
    required: true,
    unique: true,
    index: true
  },
  
  imageKitUrl: {
    type: String,
    required: true
  },
  
  thumbnailUrl: {
    type: String
  },
  
  // File properties
  mimeType: {
    type: String,
    required: true
  },
  
  size: {
    type: Number,
    required: true
  },
  
  // Image specific fields (populated by ImageKit)
  width: Number,
  height: Number,
  hasAlpha: Boolean,
  
  // Optimization data
  filePath: String,
  tags: [String],
  customMetadata: mongoose.Schema.Types.Mixed,
  
  // Status tracking
  status: {
    type: String,
    enum: ['uploading', 'processing', 'completed', 'failed'],
    default: 'uploading'
  },
  
  uploadedAt: {
    type: Date,
    default: Date.now
  },
  
  lastAccessedAt: Date,
  accessCount: {
    type: Number,
    default: 0
  }
});

// Index for queries
fileSchema.index({ userId: 1, uploadedAt: -1 });
fileSchema.index({ fileId: 1 });

module.exports = mongoose.model('File', fileSchema);

Step 4: ImageKit Configuration

 // config/imagekit.js const ImageKit = require('imagekit'); const crypto = require('crypto');

const imagekit = new ImageKit({ publicKey: process.env.IMAGEKIT\_PUBLIC\_KEY, privateKey: process.env.IMAGEKIT\_PRIVATE\_KEY, urlEndpoint: process.env.IMAGEKIT\_URL\_ENDPOINT });

// Validation function before upload const validateImage = async (fileBuffer, originalName) => { const validation = { isValid: true, errors: \[\], metadata: {} };

try { // Check file signature for magic numbers const signatures = { jpeg: \[0xFF, 0xD8, 0xFF\], png: \[0x89, 0x50, 0x4E, 0x47\], gif: \[0x47, 0x49, 0x46\], webp: \[0x52, 0x49, 0x46, 0x46\] };


const fileSignature = Array.from(fileBuffer.slice(0, 4));

let detectedType = null;
for (const [type, sig] of Object.entries(signatures)) {
  if (sig.every((byte, i) => fileSignature[i] === byte)) {
    detectedType = type;
    break;
  }
}

if (!detectedType) {
  validation.isValid = false;
  validation.errors.push('Invalid or corrupted image file');
}

// Check for malicious content (basic)
if (fileBuffer.includes('<?php') || fileBuffer.includes('<script')) {
  validation.isValid = false;
  validation.errors.push('File contains malicious content');
}

// Extract metadata using sharp (optional)
const sharp = require('sharp');
const metadata = await sharp(fileBuffer).metadata();
validation.metadata = {
  width: metadata.width,
  height: metadata.height,
  format: metadata.format,
  hasAlpha: metadata.hasAlpha,
  orientation: metadata.orientation
};

// Image size constraints
if (metadata.width && metadata.width > 5000) {
  validation.errors.push('Image width exceeds 5000px limit');
}

if (metadata.height && metadata.height > 5000) {
  validation.errors.push('Image height exceeds 5000px limit');
}


} catch (error) { validation.isValid = false; validation.errors.push(`Validation failed: ${error.message}`); }

return validation; };

// Custom transformation based on use case const getTransformations = (type = 'default') => { const transformations = { avatar: \[ { width: 200, height: 200, crop: 'fill' }, { quality: 80, format: 'webp' } \], product: \[ { width: 800, height: 800, crop: 'limit' }, { quality: 85, format: 'webp' } \], thumbnail: \[ { width: 150, height: 150, crop: 'thumb' }, { quality: 70, format: 'webp' } \], default: \[ { quality: 80, format: 'auto' } \] };

return transformations\[type\] || transformations.default; };

module.exports = { imagekit, validateImage, getTransformations };

Step 5: Upload Service with Server Cleanup

javascript

// services/uploadService.js const fs = require('fs').promises; const path = require('path'); const { imagekit, validateImage, getTransformations } = require('../config/imagekit'); const File = require('../models/File');

class UploadService {

// Main upload method async uploadToImageKitAndCleanup(localFilePath, fileMetadata, userId, options = {}) { let tempFileExists = true;

try {
  // 1. Read the local file
  const fileBuffer = await fs.readFile(localFilePath);
  
  // 2. Validate before upload
  const validation = await validateImage(fileBuffer, fileMetadata.originalname);
  
  if (!validation.isValid) {
    throw new Error(`Validation failed: ${validation.errors.join(', ')}`);
  }
  
  // 3. Prepare upload parameters
  const folder = options.folder || `users/\({userId}/\){new Date().toISOString().split('T')[0]}`;
  const tags = options.tags || ['user-upload'];
  const transformationType = options.transformationType || 'default';
  
  // 4. Upload to ImageKit
  console.log(`📤 Uploading to ImageKit: ${fileMetadata.originalname}`);
  
  const uploadResponse = await imagekit.upload({
    file: fileBuffer,
    fileName: fileMetadata.originalname,
    folder: folder,
    tags: tags,
    useUniqueFileName: true,
    isPrivateFile: options.isPrivate || false,
    customMetadata: {
      userId: userId.toString(),
      originalSize: fileMetadata.size,
      uploadedAt: new Date().toISOString(),
      ...options.customMetadata
    },
    transformation: getTransformations(transformationType)
  });
  
  console.log(`✅ Uploaded to ImageKit. File ID: ${uploadResponse.fileId}`);
  
  // 5. Generate thumbnail URL (using ImageKit transformations)
  const thumbnailUrl = imagekit.url({
    path: uploadResponse.filePath,
    transformation: getTransformations('thumbnail')
  });
  
  // 6. Store metadata in database
  const dbFile = await File.create({
    userId: userId,
    originalName: fileMetadata.originalname,
    fileId: uploadResponse.fileId,
    imageKitUrl: uploadResponse.url,
    thumbnailUrl: thumbnailUrl,
    mimeType: fileMetadata.mimetype,
    size: fileMetadata.size,
    width: validation.metadata.width,
    height: validation.metadata.height,
    hasAlpha: validation.metadata.hasAlpha,
    filePath: uploadResponse.filePath,
    tags: tags,
    customMetadata: options.customMetadata,
    status: 'completed'
  });
  
  console.log(`💾 Stored in database. File ID: ${dbFile._id}`);
  
  // 7. Delete local temp file
  await fs.unlink(localFilePath);
  tempFileExists = false;
  console.log(`🗑️ Removed local file: ${localFilePath}`);
  
  // 8. Try to remove empty parent directories
  await this.cleanupEmptyDirectories(path.dirname(localFilePath));
  
  return {
    success: true,
    file: {
      id: dbFile._id,
      fileId: uploadResponse.fileId,
      url: uploadResponse.url,
      thumbnailUrl: thumbnailUrl,
      width: validation.metadata.width,
      height: validation.metadata.height,
      size: fileMetadata.size
    },
    imageKitResponse: uploadResponse
  };
  
} catch (error) {
  // Cleanup on error
  if (tempFileExists) {
    try {
      await fs.unlink(localFilePath);
      console.log(`🧹 Cleaned up failed upload file: ${localFilePath}`);
    } catch (cleanupError) {
      console.error(`Failed to cleanup: ${cleanupError.message}`);
    }
  }
  
  console.error(`❌ Upload failed: ${error.message}`);
  throw error;
}


}

// Clean up empty directories async cleanupEmptyDirectories(dirPath) { try { const files = await fs.readdir(dirPath); if (files.length === 0) { await fs.rmdir(dirPath); console.log(`🗑️ Removed empty directory: ${dirPath}`);

 
    // Try to clean parent directory
    const parentDir = path.dirname(dirPath);
    await this.cleanupEmptyDirectories(parentDir);
  }
} catch (error) {
  // Ignore errors - directory might not be empty or doesn't exist
}


}

// Get file from ImageKit by ID (for serving) async getFileById(fileId, userId) { const file = await File.findOne({ \_id: fileId, userId: userId });

 
if (!file) {
  throw new Error('File not found or access denied');
}

// Update access tracking
file.lastAccessedAt = new Date();
file.accessCount += 1;
await file.save();

return file;


}

// Delete file from both ImageKit and database async deleteFile(fileId, userId) { const file = await File.findOne({ \_id: fileId, userId: userId });


if (!file) {
  throw new Error('File not found or access denied');
}

// Delete from ImageKit
await imagekit.deleteFile(file.fileId);
console.log(`🗑️ Deleted from ImageKit: ${file.fileId}`);

// Delete from database
await file.deleteOne();
console.log(`🗑️ Deleted from database: ${fileId}`);

return { success: true, message: 'File deleted successfully' };
 

}

// List user files with pagination async listUserFiles(userId, page = 1, limit = 20) { const skip = (page - 1) \* limit;
 
const [files, total] = await Promise.all([
  File.find({ userId: userId })
    .sort({ uploadedAt: -1 })
    .skip(skip)
    .limit(limit)
    .select('-__v'),
  File.countDocuments({ userId: userId })
]);

return {
  files,
  pagination: {
    page,
    limit,
    total,
    pages: Math.ceil(total / limit)
  }
};


} }

module.exports = new UploadService();

Step 6: Express Route Handler


// routes/upload.routes.js const express = require('express'); const multer = require('multer'); const path = require('path'); const fs = require('fs'); const uploadService = require('../services/uploadService'); const { authenticate } = require('../middleware/auth');

const router = express.Router();

// Dynamic user-based storage const createUserStorage = (req, file, cb) => { const userId = req.user.id; const timestamp = Date.now(); const tempDir = path.join(\_\_dirname, '../uploads/temp', userId, timestamp.toString());

if (!fs.existsSync(tempDir)) { fs.mkdirSync(tempDir, { recursive: true }); }

cb(null, tempDir); };

const generateSecureFilename = (originalname) => { const timestamp = Date.now(); const random = Math.random().toString(36).substring(2, 10); const ext = path.extname(originalname); const sanitized = originalname .replace(/\[^a-zA-Z0-9.-\]/g, '-') .substring(0, 40);

return `\({timestamp}-\){random}-\({sanitized}\){ext}`; };

const storage = multer.diskStorage({ destination: createUserStorage, filename: (req, file, cb) => { cb(null, generateSecureFilename(file.originalname)); } });

// File filter for validation const fileFilter = (req, file, cb) => { const allowedMimes = \['image/jpeg', 'image/png', 'image/gif', 'image/webp'\];

if (allowedMimes.includes(file.mimetype)) { cb(null, true); } else { cb(new Error('Only images are allowed (JPEG, PNG, GIF, WEBP)'), false); } };

const upload = multer({ storage: storage, limits: { fileSize: parseInt(process.env.MAX\_FILE\_SIZE) || 10 \* 1024 \* 1024 }, fileFilter: fileFilter });

// Upload endpoint router.post('/upload', authenticate, upload.single('file'), async (req, res) => { try { if (!req.file) { return res.status(400).json({ error: 'No file uploaded' }); }

```plaintext
console.log(`📁 File received: ${req.file.path}`);
console.log(`👤 User: ${req.user.id}`);
console.log(`📄 Original name: ${req.file.originalname}`);

// Upload to ImageKit, store in DB, delete from server
const result = await uploadService.uploadToImageKitAndCleanup(
  req.file.path,
  {
    originalname: req.file.originalname,
    mimetype: req.file.mimetype,
    size: req.file.size
  },
  req.user.id,
  {
    folder: req.body.folder || 'general',
    tags: req.body.tags ? req.body.tags.split(',') : ['user-upload'],
    transformationType: req.body.type || 'default',
    customMetadata: {
      source: 'web-upload',
      userAgent: req.headers['user-agent'],
      ipAddress: req.ip
    }
  }
);

res.status(201).json({
  success: true,
  message: 'File uploaded successfully',
  data: result.file
});


} catch (error) { console.error('Upload error:', error);

```plaintext
// Cleanup local file if it still exists
if (req.file && req.file.path && fs.existsSync(req.file.path)) {
  fs.unlinkSync(req.file.path);
  console.log(`🧹 Cleaned up file after error: ${req.file.path}`);
}

res.status(500).json({
  success: false,
  error: error.message
});

} });

// Get user's files router.get('/files', authenticate, async (req, res) => { try { const page = parseInt(req.query.page) || 1; const limit = parseInt(req.query.limit) || 20;

const result = await uploadService.listUserFiles(req.user.id, page, limit);

res.json({ success: true, data: result });

} catch (error) { res.status(500).json({ success: false, error: error.message }); } });

// Get single file by ID router.get('/files/:fileId', authenticate, async (req, res) => { try { const file = await uploadService.getFileById(req.params.fileId, req.user.id);

res.json({ success: true, data: { id: file._id, url: file.imageKitUrl, thumbnailUrl: file.thumbnailUrl, originalName: file.originalName, size: file.size, width: file.width, height: file.height, uploadedAt: file.uploadedAt, accessCount: file.accessCount } });

} catch (error) { res.status(404).json({ success: false, error: error.message }); } });

// Delete file router.delete('/files/:fileId', authenticate, async (req, res) => { try { const result = await uploadService.deleteFile(req.params.fileId, req.user.id);

res.json({ success: true, message: result.message });

} catch (error) { res.status(404).json({ success: false, error: error.message }); } });

module.exports = router;


Step 7: Authentication Middleware 

```plaintext
 // middleware/auth.js const jwt = require('jsonwebtoken');

const authenticate = async (req, res, next) => { try { const token = req.headers.authorization?.split(' ')\[1\];

if (!token) { return res.status(401).json({ error: 'Authentication required' }); }

const decoded = jwt.verify(token, process.env.JWT\_SECRET); req.user = { id: decoded.userId, email: decoded.email };

next();

} catch (error) { res.status(401).json({ error: 'Invalid or expired token' }); } };

module.exports = { authenticate };

Step 8: Main Application Entry

// app.js const express = require('express'); const mongoose = require('mongoose'); const dotenv = require('dotenv'); const morgan = require('morgan'); const uploadRoutes = require('./routes/upload.routes'); const { cleanupOldTempFiles } = require('./services/cleanupService');

dotenv.config();

const app = express();

// Middleware app.use(express.json()); app.use(morgan('combined'));

// Database connection mongoose.connect(process.env.MONGODB\_URI) .then(() => console.log('✅ MongoDB connected')) .catch(err => console.error('MongoDB connection error:', err));

// Routes app.use('/api/uploads', uploadRoutes);

// Health check app.get('/health', (req, res) => { res.json({ status: 'OK', timestamp: new Date().toISOString() }); });

// Scheduled cleanup for orphaned temp files (runs every hour) setInterval(() => { cleanupOldTempFiles(); }, 60 \* 60 \* 1000);

// Error handling middleware app.use((err, req, res, next) => { console.error(err.stack);

if (err instanceof multer.MulterError) { if (err.code === 'FILE\_TOO\_LARGE') { return res.status(413).json({ error: 'File too large' }); } return res.status(400).json({ error: err.message }); }

res.status(500).json({ error: 'Internal server error' }); });

const PORT = process.env.PORT || 3000; app.listen(PORT, () => { console.log(`🚀 Server running on port ${PORT}`); });

module.exports = app;

Step 9: Cleanup Service for Orphaned Temp Files

// services/cleanupService.js const fs = require('fs').promises; const path = require('path');

const cleanupOldTempFiles = async (maxAgeHours = 24) => { const tempDir = path.join(\_\_dirname, '../uploads/temp');

try { const users = await fs.readdir(tempDir); const now = Date.now(); let deletedCount = 0;

for (const user of users) { const userDir = path.join(tempDir, user); const sessions = await fs.readdir(userDir);

for (const session of sessions) { const sessionDir = path.join(userDir, session); const stats = await fs.stat(sessionDir); const ageHours = (now - stats.mtimeMs) / (1000 \* 60 \* 60);


if (ageHours > maxAgeHours) {
  await fs.rm(sessionDir, { recursive: true, force: true });
  deletedCount++;
  console.log(`🧹 Removed old temp session: ${sessionDir}`);
}


}

// Remove empty user directory const remainingFiles = await fs.readdir(userDir); if (remainingFiles.length === 0) { await fs.rmdir(userDir); } }

if (deletedCount > 0) { console.log(`✅ Cleaned up ${deletedCount} old temp sessions`); }

} catch (error) { // Temp directory might not exist if (error.code !== 'ENOENT') { console.error('Cleanup error:', error); } } };

module.exports = { cleanupOldTempFiles };

Storing Uploaded Files and Serving Them in Express

Where Uploaded Files Are Stored

Memory Storage vs Disk Storage

Memory Storage Example

File Storage Folder Structure

Recommended Folder Structure

Creating This Structure Programmatically

Dynamic File Organization by User

Complete Pipeline: Upload → Validate → ImageKit → Database → Cleanup

Step 1: Install Dependencies

.env

Step 3: Database Models

More from this blog

Monorepo Arcitechture

KAFKA

Websocket - Redis

Open IDC OAUTH

Command Palette

Where Uploaded Files Are Stored

Memory Storage vs Disk Storage

Memory Storage Example

File Storage Folder Structure

Recommended Folder Structure

Creating This Structure Programmatically

Dynamic File Organization by User

Complete Pipeline: Upload → Validate → ImageKit → Database → Cleanup

Step 1: Install Dependencies

.env

Step 3: Database Models

More from this blog