Using Arrow keys to cycle through Mozilla Screenshots



 The Mozilla-Services' screenshots add-on for the Mozilla Firefox browser is a nifty browser add-on that allows you to snip a portion of your browser's screen to create a screenshot that you can then either download to your computer, or host on Mozilla's servers. It also comes with some simple image editing tools to allow you to draw on top of your screenshot, etc.. Screenshots consists of both the add-on itself, and the server that it communicates with. The add-on allows you to capture parts of your browser's window and then save the resulting screenshot to your computer or the cloud. The server provides a UI to browse, edit, and download these screenshots after they have been uploaded by the add-on.

The server is written with React, and NodeJs with express routing, and uses a Postgre SQL database to store screenshots and their metadata. I believe the add-on uses IndexedDB to store the current screenshot before it is either downloaded or saved to the cloud.

The part I would be working on for my issue would be the server. The task was to allow the use of  left and right arrow keys to transition between screenshots that had been stored on the server when you are viewing them individually.

An individual shot view with  newly coded links to transition between images. Pressing the left and right arrow keys also activates the next or previous links appropriately.



It required (more or less) the following steps for me to set up the code environment:

  • download and install the latest version of Firefox developer edition
  • Install the Postgresql server and service, and related DBMS with the help of these instructions for an Ubuntu machine
    • o   Create a file to hold a reference to to hold the apt repo:
      > sudo touch /etc/apt/sources.list.d/pgdg.list
      o   Add this line to the new file:
      deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main
      o   Get the key for the apt-get repo:
      wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add –
      o   Sudo apt-get update
      o   Install postgre with the apt repo:
      sudo apg-get install psgtresql-11 pgadmin4
      o   Add yourself as a user so you can actually use the damn ‘psql’ command:
      sudo -u postgres createuser --superuser $USER
      o   Create a database under your username:
      sudo -u postgres createdb $USER
      o    
      ·         Install the postgres client:
      o   sudo apt-get install postgresql-client
       
     
  •  After pulling the repo, running npm intall to pull in the required dependencies yielded the following error: npm ERR! 404 Not Found: har-validator@5.1.2
    • To solve this issue I tried:
      • running npm update
      • uninstalling, and reinstalling the har-validator via npm in the hopes that it could find a valid version, but trying to uninstall gave the same 404 error as trying to install the package did
      •  Tried installing and using yarn cli because there is a yarn package for har-validator which may be mismatched with the npm version ( see this github issue)
      • tried blowing away the package-lock.json file, and allowing it to re-generate via the package.json file (this may have been inadvisable, but was an attempt to get anything to work)
     
Since none of that worked, I continued on with the severity of my tinkering by manually inspecting and editing the contents of package.json. Within, I found various references to har-validator@5.1.3, which this github issue had suggested was messing things up. As a quick fix, rather than allowing npm's automation to try to grab the most recent (and broken) package, I edited those instances back to 5.1.2, and npm finally decided to make it through the package installation process.

When I tried to build and run the repo's server, however, I found I had to manually install babel-cli, browserify, node-uglify, and eslint. This may have been due to my obliteration of the package-lock.json file.


Neat. Now the  server would run, but would complain that it couldn't access the database. I wanted to get the database to run without the need for passwords since it would never see the light of day beyond localhost, and tried to edit its pg_hba.conf file to make this happen, but the server wasn't having it. Eventually I had to manually set the password to my database so that the server would be able to access the newly created tables.

If you don't choose to run the code repo's addon (./bin/run-addon), trying to use the addon in the mozilla browser at the server's address (localhost:10080), would cause the browser to redirect to Mozilla's live server to serve pages for viewing you screenshots, and try to store all the screenshots you took on the cloud instead of the development database.

Since the Screenshots addon also uses IndexedDB, you need to change Firefox's permission by going to about:config in its URL bar, and enabling various dom.indexedDB features.

be sure to enable IndexedDB in Firefox Developer Edition


Once all that was set up, it was time to debug. I've had success attaching debuggers to servers running React before in VS Code, but that was with using the Chrome Debugger extension. Since the Screenshots add-on was for Firefox, this would be out of the question.  Unfortunately, the Screenshots code repo only talked about debugging the add-on itself via about:debugging (the URL bar to bring up a page on debugging). This featured a "learn more" link that only seemed to talk about how to run the add-on, not how to debug it in the context of a server.

This information is quite informative... however I wanted to work on the server the extension talked to, not the extension itself.
 Well, the server ran via NodeJS, and used express routing, so perhaps I could attach the VS Code IDE to the server process after it was started with node. The server was actually started up using a bash script file called "run-addon", so I went into that and added the --inspect flag to its call to run the server with node, and then tried to edit VS Code's launch.json file, which defines how it launches its debugger, to try to attach the debugger to port 9229 of a my running server. 9229 is the default port that Node will try to talk to debuggers on if you start a server with the --inspect option.



This didn't yield any positive results, nor did trying to install a mozilla debugger extension I found for VS Code, so it was time to use my old friend, console.log().

 Since I didn't really know how the server actually worked I used console.log() on everything, and eventually found a database query that would load a screenshot to the individual screenshots view. I added some logic to that to pull in the URLs for all screenshots stored in the database so I could then determine what screenshots came before and after the one that would be currently displayed in the user's browser.
 Since the React app used a vaguely MVC style model, where a file called view.js  held a React Component, that had properties delivered to it via model.js (which I think had structures for data for the client and the server to map to and from the view), which was associated with a controller file, and eventually led back to server-shot.js, which pulled in screenshot objects from the Postgre database.I added some properties to the React component and its related model to hold the next and previous urls, and added some on screen links to allow users to traverse these shots.

This didn't really do anything to get arrow keys working, so I then had to add an event listener to check if the user hit the left or right arrow key. This proved to be a bit more difficult than anticipated, because React views are written with JSX, and they don't really like you inserting <script> tags to add your listeners inline with your page code. Trying to add a listener to redirect to the next and previous image in the react component's componentDidMount() callback function didn't really help because the properties that would be needed to determine the url to redirect to when the user pressed a key would not be populated at the time.

This doesn't work since this.props.prevShotId is undefined when the component mounts:


window.addEventListener("keypress", (event)=>{
          const keyName = event.key;
          if(keyName == "ArrowLeft"){
            window.location.replace(this.props.prevShotId);
          }
          if (keyName == "ArrowRight"){
            window.location.replace(this.props.prevShotId);
          }
});

however giving the previous and next image links I had already made unique id attributes, and replacing window.location.replace() with document.getElementById().click() did.


  So after all that messing about, I managed to get the Mozilla Screenshots server to transition between all of the shots in the database by using the arrow keys on your keyboard.

Comments

Post a Comment

Popular posts from this blog

Tinkering with Chrome Headless to Handle Mic Input

This pull request consisted of performing more research for the CodeTalker repository. One of the ongoing issues with the repository is figuring out how to make it easy to capture microphone audio across a variety of operating systems, and then process it in a way that is easy to set up across multiple user's devices (all of which have may use different operating systems). In an earlier attempt to make the development environment more accessible to potential contributors to the CodeTalker repo, I tried to mount all of the scripts that processed microphone audio to a Docker container. The advantage of this was that the Dockerfile, which defines how a docker image (virtual machine) is built can be configured to install and configure various dependencies. These dependencies in this case would be for processing microphone audio (Python libraries, and various microphone hardware handling libraries). This worked great for Unix style machines, because the microphone hardware (and other...
Read more

Research and issues with cross platform compatibility

I needed to continue to do some research on how to try to get speech-to-text functionality working in Visual Studio Code. I did some research that can be found in this pull request .  One of the issues with trying to grab microphone input and send it to an application that works across a variety of platforms is that it will not necessarily be straightforward to grab the microphone audio from various computers' hardware, and to then use a program that works across multiple platforms to parse the audio. Web apps have generally tended to be associated with the ability to run across a variety of platforms as they work on one system remotely, and send data back through some reasonably universal format like html, JavaScript, JSON, or common file formats. Things like the Web Speech API are built into some browsers, but only a few, and Google seems to make a point of making a complex and valuable feature like this unavailable in Chromium, where one might try to look at what they do t...
Read more