0

I need to serve the only static file on webserver, replacing the placeholder with $value with some conditions.

  • $value is decimal or hexadecimal text string (ex, abba, e1ee7, 01023)
  • $value is derived from client IPv4 or IPv6
  • $value is different for different client IP addresses and distributed evenly across some range
  • $value can repeat for different IP addresses, but there must be at least 5 unique values
  • $value is reproducible for each client IP address
  • $value must fit 48bit (6 bytes) when parsed from text to string
  • $value different values may not be same length
  • $value must not be empty string
  • file must be served as fast as possible

So if there were only IPv4 addresses, I'd set a $value to last digit of client IPv4 address, or last octet of it - it is enough for my needs. But the problem is IPv6.

The main idea is to make something like this (nginx way, fake code):

location = /file.dat {
    #this way
    set $placeholder_value crc32($binary_remote_addr); #  this is not real code
    #or this way
    set $placeholder_value substring(hex($binary_remote_addr), 0, 12); # this is not real code
    #or this way
    set $placeholder_value modulo($remote_addr, 10); # this is not real code


    sub_filter_types *;
    sub_filter_once on;
    sub_filter 'placeholder' '$placeholder_value';
    try_files /template.dat =404;
}

I and ChatGPT found a perl solution for nginx calculating md5 from remote_addr and taking 6 bytes of it:

  http {
    perl_modules perl/lib;
    
    perl_set $remote_addr_md5_48bit_hash '
        use Digest::MD5 qw(md5_hex);
        sub {
            my $r = shift;
            my $ip = $r->remote_addr;
            my $full_hash = md5_hex($ip);
            my $binary_hash = pack("H*", $full_hash);
            my $truncated_hash = substr($binary_hash, 0, 6);
            return unpack("H*", $truncated_hash);
        }
    ';
    
    server {
        ...
        location = /file.dat {
            set $placeholder_value $remote_addr_md5_48bit_hash;
    
            sub_filter_types *;
            sub_filter_once on;
            sub_filter 'placeholder' '0x$placeholder_value';
            try_files /template.dat =404;
        }
    }
  }

But I don't like this because of:

  • Too much perl lines here
  • I don't like defining anything outside of server.

Any advice?

I have some extra ideas but IDK if they are possible:

  • upstream, proxy_pass, hash $remote_addr and few locations with static sub_filter: requires multiple server {}, as there is no way to use location in upstream
  • transform $binary_remote_addr to hex string and use only last as $value
  • get rid of nginx and write tiny microservice in Go for this task

I made an IPv4-only solution fit my needs: Looking towards making it IPv6

server {
    set $seed_placeholder '0xDEF_5EED'; # "DEFault SEED"
    set $tepmlate_file_name '/template.dat';
    set $template_file_mime_type 'text/plain'; 
    ...

    location ~* \.dat$ {
        rewrite ^ '/int/transform-ip/$remote_addr';
    }

    location /int {
        internal;
        
        location /int/transform-ip {
            location ~ ^/int/transform-ip/(?<a>[0-9]+)\.(?<b>[0-9]+)\.(?<c>[0-9]+)\.(?<d>[0-9]+)$ {
                rewrite ^ /int/send-template/$a$b$c$d last;
            }
        }
        
        location /int/send-template {
            location ~ ^/int/send-template/(?<seed>(?:0x|)[0-9a-f+\_]+)$ {
                default_type $template_file_mime_type;
                sub_filter_types *;
                sub_filter_once on;
                sub_filter $seed_placeholder $seed;
                try_files $tepmlate_file_name =404;
            }
        }
    }

    location / {
        return 404 "other";
    }
}

0

Browse other questions tagged or ask your own question.