Saturday Update: It Lives.

No updates since Tuesday, but I have been busy.

• Monday: Rebuild the NAS and install PHP and Apache. This will give me a place to work. Done!
• Tuesday: Hello world web service. This will let me get my feet wet with building and deploying the service. Done!
• Wednesday: Architect services. I need to sit down and really think through how to implement this, since this will drive everything else downstream. Done!
• Thursday: Rethinking database schema. This needs to be done to properly store all of the data. Right now PART has data in CSV files where it uses the stock ticker as the key, but in the dividend database a GUID is used for all records. So I need to re-think how this all ties together.
• Friday: Rebuild my development environment. I am currently using Eclipse Luna which is more than 5 versions behind. I think some of the issues I am having are due to me running such an old environment, so I’m just going to rebuild from scratch. This isn’t really needed to build services but it will be needed if I want to update PART to leverage the services. Done!

High Level Architecture

I’ve managed to figure out the high level architecture of what I’d like to do. From a use-case perspective:

• Add price a price
• Fetch price(s)
• Update price(s)
• Delete price(s)

One of the key considerations is a unique key for everything. A price is typically a 3-tuple of ticker, date, and price. The challenge is with ticker. Tickers are not guaranteed to be unique on a stock exchange over the life of the exchange, since a company may de-list, and another company may join the exchange years later using the same ticker. Also, tickers are not guaranteed to be unique across exchanges. The TSX and US exchanges both have ticker symbol “T”, Telus in Canada, and AT&T in the US. Even worse, Magna, which is MG in Canada and MGA in the US, but they are the same company! So using the ticker isn’t a safe bet, and for that reason we use our own internal GUID to for companies; we call this internal id LIQ_CompanyID. This also facilitates the use-case when companies change their ticker, since if we reference the LIQ_CompanyID, we don’t care what the ticker is.

The approach above is fair enough, but if we use LIQ_CompanyID as our reference, we need a helper function to convert a current ticker symbol to the LIQ_CompanyID. Oh, and it would be nice to make this reverse lookup scalable (e.g. reverse lookup from other sources such as CUSIP or ISIN).

I also hem and hawed about multiple functions for single or multi-element lookups, or just one function which–if only one result was available–returned an array of a single object. I’ve settled on the latter (multi-element), since this would reduce server calls.

Finally, I want this to be as universal as possible, and for that reason I’m using JSON-RPC 2.0.

With all that said, the functions:

• reverseCompanyIDLookup
• addPrices
• getPrices
• updatePrices
• getPricesMetadata
• deletePrices

Database

Sadly I haven’t gotten to this yet…

Building the Development Environment

So, this was challenging, but I had success on Friday, and managed to stay on track. I’m super stoked that I managed to get a working Hello World environment up and running which let me step through my code real-time. I ended up using Eclipse PDT, and Docker, to get everything up and running. Prior to Thursday I’d never even heard of Docker, but after finding it, this thing kicks ass! Never having dealt with PHP I submersed myself in blog articles and guides on IDEs and Xdebug.

I started reading this article on How To Build a Local Development Environment Using Docker, which introduced me to the whole concept of Docker and containers for a development environment. I spent a good part of the day playing with Docker itself, and running through the introduction guide (i.e. ignoring the PHP development environment), before I even tried to string together a development environment.

The actual instructions I used were written by Tarun Lalwani, and can be found at in his article Debugging PHP web applications using XDebug inside Docker, which may be found at this link. Following his instructions I was able to get an entire environment up and running using PHP-FPM 7.3. The “gotchya” I had when doing this is that Xdebug expected port 9000 to be used on the remote debugging machine, but on my development machine that is where Squeezebox server lives, so I moved the Xdebug remote port to 9009. I also remapped Xdebug on port 9000 of the Docker image to port 9010 on the local machine.

version: '2'
services:
  nginx:
    image: nginx:latest
    volumes:
      - ./app.conf:/etc/nginx/conf.d/app.conf
      - ~/dev/phptestproject:/var/www/html

    ports:
      - "8080:80"
    container_name: nginx

  php-fpm:
    build: .
    # image: php:7.3-fpm # php-fpm:5.6-fpm
    volumes:
      - ~/dev/phptestproject:/var/www/html
    container_name: php
    ports:
      - "9010:9000"
    

High level steps:

1. Read How To Build a Local Development Environment Using Docker to give some broad background concepts on what we’re trying to do.
2. Download and install Docker.
3. Run through the docker getting started guide.
4. Install Eclipse PDT. (Version 2020-03 as of this writing; I also had to update to the latest JDK and JRE on my machine).
5. Run through the article Debugging PHP web applications using XDebug inside Docker.
6. Build the Nginx and PHP-7.3 containers, and get them up and running.
7. Point Eclipse PDT’s Xdebug to port 9009.
   

Doing that, I was able to get my Nginx and PHP containers up and running using the command docker-compose up -d, and set a breakpoint in my code, and navigate over to 127.0.0.1:8080, and have a fully functioning environment!

Next steps

I’m going to spend next week building up the test database, some JSON-RPC stubs, and then start fleshing out the code!