# Common Errors and Solutions

This document catalogs common errors encountered during development and their solutions.

---

## Error 1: Product Weight Not Persisting Across Components

### Date Encountered
2025-10-25

### Symptoms
- Weights updated via invoice upload appear correct in invoice calculator
- BUT same weights show old/incorrect values in:
  - Bulk Editor
  - Edit Product Modal
- Database actually contains correct weight values
- Issue persists even after saving

### Root Cause
**Three components were using stale in-memory data instead of fetching from database:**

1. **Edit Product Modal** (`admin.js` line 139):
   ```javascript
   function editProduct(id) {
       const product = products.find(p => p.id === id);  // ← Using stale global array!
   ```

2. **Bulk Editor Load** (`admin.js` line 564):
   ```javascript
   function loadProductsToGrid() {
       bulkEditorData = products.map(...);  // ← Using stale global array!
   ```

3. **Bulk Editor Save** (`admin.js` lines 834-842):
   - Was NOT sending `weight_kg` or `unit_type` to backend at all!

4. **Backend Bulk Update** (`bulk-import.js` lines 80-91):
   - UPDATE query didn't include `weight_kg` or `unit_type` columns

### Solution

**1. Made Edit Modal Fetch Fresh Data** (`admin.js` line 139):
```javascript
async function editProduct(id) {
    // ALWAYS fetch fresh data from database before editing
    await loadProducts();

    const product = products.find(p => p.id === id);
    // ... rest of code
}
```

**2. Made Bulk Editor Fetch Fresh Data** (`admin.js` line 564):
```javascript
async function loadProductsToGrid() {
    // ALWAYS fetch fresh data from database (single source of truth)
    await loadProducts();

    bulkEditorData = products.map(...);
    // ... rest of code
}
```

**3. Added Weight to Bulk Save Payload** (`admin.js` lines 838-839):
```javascript
products: existingProducts.map(row => ({
    id: row.id,
    cod_produs: row.cod_produs,
    cod_producator: row.cod_producator,
    weight_kg: parseFloat(row.weight_kg),      // ← ADDED
    unit_type: row.unit_type,                  // ← ADDED
    purchase_cost: parseFloat(row.purchase_cost),
    // ... rest
}))
```

**4. Updated Backend to Save Weight** (`bulk-import.js` lines 71-74, 88-89):
```javascript
// Extract from request
const weightKg = product.weight_kg !== undefined ?
    parseFloat(product.weight_kg) : existing.weight_kg;
const unitType = product.unit_type !== undefined ?
    product.unit_type : existing.unit_type;

// Include in UPDATE query
const sql = `
    UPDATE products
    SET cod_produs = ?,
        cod_producator = ?,
        weight_kg = ?,      // ← ADDED
        unit_type = ?,      // ← ADDED
        base_purchase_cost = ?,
        // ... rest
    WHERE id = ?
`;
```

### Verification
Created `check-product.js` script to query database directly:
```bash
node check-product.js
```

Confirmed that:
- ✅ Database has ONLY ONE weight field: `weight_kg`
- ✅ Invoice upload correctly saves to `weight_kg`
- ✅ Bulk editor now correctly saves to `weight_kg`
- ✅ Edit modal now correctly saves to `weight_kg`
- ✅ All components fetch fresh data before displaying

### Prevention
- **Single Source of Truth**: SQLite database, `products.weight_kg` column
- **Always fetch fresh data** before editing/displaying
- **Never trust in-memory arrays** - they can be stale
- **Test all CRUD paths** - create, read, update, delete
- **Verify database state** using direct queries when debugging

### Watch Out For
- Other fields that might have same issue (purchase_cost, markup_percentage, etc.)
- Browser caching old JavaScript after code changes (always Ctrl+F5)
- Multiple tabs with stale data
- Race conditions when multiple async loadProducts() calls happen

---

## Error 2: Invoice Parser Skipping Products with European Number Format

### Date Encountered
2025-10-25

### Symptoms
- Parser was skipping certain products from LAURENCE invoices
- Specifically affected products with prices containing thousands separators
- Example: "Kunefe" with net price "1.881,00" was not being extracted
- Console showed: `✗ SKIPPED line 175 (looks like product but didn't match pattern)`

### Root Cause
The regex pattern `[\d,]+` for matching numeric values only accepted digits and commas. When a number contained a dot as a thousands separator (European format: `1.881,00`), the pattern would only match up to the dot, causing the entire line to fail validation.

**Example Problem**:
```
Invoice line: "01-05-0083 Kunefe 418,00 4,50 0,00 1.881,00 Pcs 19,00"
                                                   ^---- Dot here breaks the regex
```

### Solution
1. **Updated Regex Pattern** (`routes/bulk-import.js` line 208):
   ```javascript
   // BEFORE
   const productLinePattern = /^(\d{2}-\d{2}-\d{4})\s+(.+?)\s+([\d,]+)\s+([\d,]+)\s+([\d,]+)\s+([\d,]+)\s+(Kg|Pcs)\s+([\d,]+)/;

   // AFTER
   const productLinePattern = /^(\d{2}-\d{2}-\d{4})\s+(.+?)\s+([\d,.]+)\s+([\d,.]+)\s+([\d,.]+)\s+([\d,.]+)\s+(Kg|Pcs)\s+([\d,.]+)/;
   ```
   Changed `[\d,]+` to `[\d,.]+` to accept both dots and commas.

2. **Created parseNumber Function** (lines 226-228):
   ```javascript
   const parseNumber = (str) => {
       return parseFloat(str.replace(/\./g, '').replace(',', '.'));
   };
   ```
   This function:
   - Removes all dots (thousands separators): `"1.881,00"` → `"1881,00"`
   - Replaces comma with dot (decimal point): `"1881,00"` → `"1881.00"`
   - Parses as JavaScript float: `1881.00`

3. **Applied to All Numeric Fields** (lines 230-233):
   ```javascript
   const quantity = parseNumber(qty);
   const totalNetPrice = parseNumber(netPrice);
   const perUnitPrice = quantity > 0 ? totalNetPrice / quantity : parseNumber(price);
   ```

### Prevention
- **Always consider international number formats** when parsing financial data
- **Test with real invoice samples** that contain various number formats
- **Add console logging** for skipped lines to catch parsing failures early

### Watch Out For
- Other European formats: spaces as thousands separators (`1 881,00`)
- Different decimal separators in different countries
- Mixed formats in the same invoice (unlikely but possible)
- Very large numbers that might have multiple thousands separators (`1.234.567,89`)

---

## Error 2: Server Changes Not Taking Effect After Code Modifications

### Date Encountered
2025-10-25

### Symptoms
- Modified parser code but changes didn't take effect
- Products still being skipped despite regex fix
- Server appeared to be running old code

### Root Cause
Node.js caches required modules. When backend code is modified, the server must be restarted to reload the updated code. The server was still running with the old version of `routes/bulk-import.js`.

### Solution
1. **Kill the existing Node process**:
   ```bash
   # Find process using port 3000
   netstat -ano | findstr :3000

   # Kill the process (replace PID with actual process ID)
   taskkill //F //PID <PID>
   ```

2. **Restart the server**:
   ```bash
   cd "c:\Web Projects\chocolate-ordering-system"
   npm start
   ```

3. **Verify changes** by checking server console logs for new parsing behavior

### Prevention
- **Use nodemon** for automatic server restart on file changes:
  ```bash
  npm install --save-dev nodemon
  ```
  Update `package.json`:
  ```json
  "scripts": {
    "start": "node server.js",
    "dev": "nodemon server.js"
  }
  ```
  Then run: `npm run dev`

- **Always restart server** after modifying backend code
- **Check process IDs** before killing to avoid killing wrong process
- **Look for success indicators** in console after restart

### Watch Out For
- Multiple Node processes running (can cause port conflicts)
- Changes to frontend files (HTML/CSS/JS in `public/`) don't require server restart, only browser refresh
- Changes to route files, database.js, server.js all require server restart

---

## Error 3: Browser Cache Showing Old UI After HTML/CSS Changes

### Date Encountered
Multiple times during development

### Symptoms
- Modified HTML or CSS but changes not visible in browser
- User reports "instructions are still there" after deletion
- UI appears unchanged despite code updates

### Root Cause
Browsers aggressively cache static assets (HTML, CSS, JavaScript) for performance. When files are updated on the server, the browser may still serve the old cached version.

### Solution
**Hard Refresh** to bypass cache:
- **Windows**: `Ctrl + F5` or `Ctrl + Shift + R`
- **Mac**: `Cmd + Shift + R`
- **Chrome DevTools**: Right-click refresh button → "Empty Cache and Hard Reload"

### Prevention
- **Instruct users** to always hard refresh after updates
- **Add cache-busting** to static assets:
  ```html
  <link rel="stylesheet" href="/css/style.css?v=1.0.1">
  <script src="/js/admin.js?v=1.0.1"></script>
  ```
  Increment version number after changes.

- **Set cache headers** in server.js:
  ```javascript
  app.use(express.static('public', {
      maxAge: 0,  // Don't cache during development
      etag: false
  }));
  ```

### Watch Out For
- Service workers can also cache aggressively
- Mobile browsers cache more aggressively than desktop
- Multiple browser tabs may have different cache states

---

## Error 4: Dynamic Summary Not Updating After Table Edits

### Date Encountered
2025-10-25 (potential issue, prevented proactively)

### Symptoms
- Table values change but summary box shows old totals
- Weight/quantity edits don't update "Total Weight" display
- Price edits don't update "Total Invoice Cost"

### Root Cause
Summary calculations were only run once during initial display. Subsequent edits to the table didn't trigger recalculation of summary metrics.

### Solution
**Call updateTotalCalculatedPrice() after any data modification**:

1. **After recalculating all transport costs** (admin.js line 1665):
   ```javascript
   function recalculateAllTransportCosts() {
       // ... calculation logic
       updateRowDisplay(index);
   });

   // Update total calculated price
   updateTotalCalculatedPrice();  // ← Added this
   }
   ```

2. **After recalculating single row** (admin.js line 1728):
   ```javascript
   function recalculateSingleRowTotals(rowIndex) {
       // ... calculation logic
       updateRowDisplay(rowIndex);

       // Update total calculated price
       updateTotalCalculatedPrice();  // ← Added this
   }
   ```

3. **Make updateTotalCalculatedPrice() comprehensive**:
   ```javascript
   function updateTotalCalculatedPrice() {
       // Recalculate ALL metrics, not just total price
       let totalRawCostEUR = 0;
       let totalItems = 0;
       let totalWeight = 0;
       let totalBoxes = 0;

       // ... calculate from items

       // Completely re-render summary HTML
       summaryDiv.innerHTML = `...`;
   }
   ```

### Prevention
- **Always update dependent displays** after data changes
- **Use event-driven architecture** for cascading updates
- **Centralize display update logic** in single function
- **Test all edit scenarios**: quantity, weight, price, markup, VAT

### Watch Out For
- Circular update loops (A updates B which updates A)
- Performance issues if recalculating too frequently
- Stale references to DOM elements after re-render
- Focus loss when re-rendering editable cells

---

## Error 5: querySelector Failing After Dynamic HTML Update

### Date Encountered
2025-10-25 (potential issue, handled proactively)

### Symptoms
- `querySelector()` returns null after updating `innerHTML`
- Event listeners stop working after re-render
- "Cannot read property 'textContent' of null" errors

### Root Cause
When you set `innerHTML`, all existing DOM elements are destroyed and recreated. Any references to old elements or attached event listeners are invalidated.

### Solution
1. **Update specific elements** instead of wholesale innerHTML replacement:
   ```javascript
   // BAD: Replaces entire summary, losing any references
   summaryDiv.innerHTML = `<div>...</div>`;

   // BETTER: Update specific text nodes
   document.getElementById('summaryTotalWeight').textContent = totalWeight.toFixed(3) + ' kg';
   ```

2. **Re-query elements after innerHTML update**:
   ```javascript
   summaryDiv.innerHTML = `...`;
   // Old references are now invalid
   const totalPriceDiv = summaryDiv.querySelector('.total-price');  // Query again
   ```

3. **Use data attributes for reliable selection**:
   ```html
   <div data-metric="total-weight">0 kg</div>
   ```
   ```javascript
   summaryDiv.querySelector('[data-metric="total-weight"]').textContent = ...;
   ```

### Prevention
- **Minimize innerHTML usage** for dynamic content
- **Prefer targeted updates** to specific elements
- **Use event delegation** for dynamically created elements:
  ```javascript
  // Instead of attaching to each cell
  table.addEventListener('blur', (e) => {
      if (e.target.dataset.field === 'price') {
          updatePrice(e.target);
      }
  }, true);
  ```

### Watch Out For
- Memory leaks from orphaned event listeners
- Focus loss during editing when innerHTML updates
- Animation/transition interruption during re-render

---

## Best Practices Summary

### 1. Number Parsing
- Always consider international formats
- Use helper functions for parsing
- Test with real-world data

### 2. Server Restart
- Restart after backend code changes
- Use nodemon for auto-restart
- Check console for confirmation

### 3. Browser Cache
- Hard refresh after frontend changes
- Use cache-busting in production
- Set appropriate cache headers

### 4. Dynamic Updates
- Update all dependent displays
- Centralize calculation logic
- Test all edit scenarios

### 5. DOM Manipulation
- Prefer targeted updates
- Re-query after innerHTML changes
- Use event delegation

---

## Debugging Checklist

When something doesn't work:

1. ☑ **Check browser console** for JavaScript errors
2. ☑ **Check server console** for backend errors
3. ☑ **Hard refresh browser** (Ctrl+F5)
4. ☑ **Restart server** if backend code changed
5. ☑ **Check network tab** for failed requests
6. ☑ **Verify data format** (especially numbers, dates)
7. ☑ **Check element existence** before accessing properties
8. ☑ **Review recent changes** - last working state vs current
9. ☑ **Test with real data** - edge cases matter
10. ☑ **Check for typos** - especially in variable names, IDs

---

## Error 6: 500 Internal Server Error After Deployment (Missing Dependencies)

### Date Encountered
2025-10-27

### Symptoms
- Site returns HTTP 500 error after deploying to cPanel
- Passenger error: "Web application could not be started"
- Manual server start shows: `Error: Cannot find module 'express'`
- All code files deployed correctly via git clone

### Root Cause
After cloning the repository from GitHub, **npm dependencies were not installed**. The git repository only contains `package.json`, not the `node_modules/` directory. Without running `npm install`, the application has no access to required packages (express, sqlite3, bcrypt, etc.).

**Common Scenario**: Developer deploys code but forgets the `npm install` step.

### Solution

**Step 1: Install Dependencies**
```bash
cd /home/username/reseller.methexis.eu
source /home/username/nodevenv/reseller.methexis.eu/22/bin/activate
npm install
```

**Note**: npm install can take 20-30 minutes on shared hosting (254 packages in this project).

**Step 2: Pre-initialize Database**
The database initialization includes slow operations (bcrypt hashing, many ALTER TABLE commands) that can cause Passenger to timeout during startup. Initialize it manually first:

```bash
# Run server briefly to create database.db
timeout 30 node server.js &
sleep 10
# Kill the process
pkill -f "node server.js"
```

This creates the `database.db` file (64KB) with tables and default users.

**Step 3: Restart Passenger**
```bash
touch tmp/restart.txt
```

Wait 5-10 seconds, then test the site.

### Verification
```bash
# Check dependencies installed
ls -la node_modules/ | wc -l  # Should show 250+ directories

# Check database created
ls -lh database.db  # Should exist and be ~64KB

# Check site responds
curl -I http://yoursite.com  # Should return HTTP 200 OK
```

### Prevention
- **Add to deployment checklist**: Always run `npm install` after git clone/pull
- **Document in README**: Add clear deployment instructions
- **Use deployment scripts**: Automate the full deployment process
- **Pre-initialize database locally**: Commit optimized database.js that doesn't run slow operations every startup

### Watch Out For
- **Slow npm install on shared hosting**: Can take 25+ minutes (be patient!)
- **Passenger startup timeout**: Database initialization with bcrypt can timeout
- **Port conflicts**: If manually testing, kill processes before letting Passenger start
- **Missing .htaccess**: Passenger configuration required for CloudLinux hosting
- **Node version mismatch**: Ensure cPanel Node.js version matches development environment

### Passenger-Specific Notes
cPanel uses **Passenger (CloudLinux)** to run Node.js apps, not PM2:
- `.htaccess` contains Passenger configuration
- Restart with: `touch tmp/restart.txt`
- No need for PM2 installation
- Passenger has startup timeout (~20 seconds) - keep initialization fast

---

## Error 7: Passenger Startup Timeout with Database Initialization

### Date Encountered
2025-10-27 (related to Error 6)

### Symptoms
- Passenger shows: "Web application could not be started"
- `database.js` runs slowly with bcrypt.hashSync() calls
- App works when run manually but fails with Passenger
- Error ID in Passenger error page

### Root Cause
The `database.js` initialization runs **synchronously** every time the app starts:
1. Creates tables (fast)
2. Runs many `ALTER TABLE` commands with empty callbacks (slow)
3. Hashes admin password with `bcrypt.hashSync()` - takes ~2 seconds
4. Hashes customer password with `bcrypt.hashSync()` - takes ~2 seconds
5. Total initialization: 5-10 seconds

Passenger has a startup timeout of ~20 seconds. On slow shared hosting, this can cause timeouts.

### Solution

**Quick Fix**: Pre-create database before Passenger starts (see Error 6)

**Long-term Fix**: Optimize database.js initialization:

1. **Check if tables exist before altering**:
   ```javascript
   db.get(`PRAGMA table_info(products)`, (err, row) => {
       if (!row || !row.internal_sku) {
           db.run(`ALTER TABLE products ADD COLUMN internal_sku TEXT UNIQUE`);
       }
   });
   ```

2. **Hash passwords once, store hash in code**:
   ```javascript
   // Instead of hashing on every startup:
   const adminPassword = bcrypt.hashSync('Methexis1234', 10);

   // Store pre-computed hash (generated once offline):
   const adminPasswordHash = '$2b$10$xyz...';  // bcrypt.hashSync('Methexis1234', 10)

   db.run(`INSERT OR IGNORE INTO users (email, password, is_admin)
           VALUES (?, ?, ?)`, ['office@methexis.eu', adminPasswordHash, 1]);
   ```

3. **Use async initialization**:
   ```javascript
   async function initializeDatabase() {
       await createTables();
       await seedDefaultData();
       console.log('Database ready');
   }
   ```

### Prevention
- Keep database initialization fast (<5 seconds)
- Run expensive operations once, not on every startup
- Use migrations for schema changes instead of ALTER TABLE on startup
- Pre-create database for production deployments

---

## Getting Help

If you encounter a new error:
1. **Document the error** - exact error message, stack trace
2. **Note the symptoms** - what you expected vs what happened
3. **Identify the trigger** - what action caused it
4. **Check this document** - has it happened before?
5. **Add to this document** - if it's new, document the solution