Module By Module: Hints

Nota Bene: The pace of progress is quite fast in this area so you might find quite a few changes in things like capabilities and user interfaces. In general, please do remember that these hints are intended to be little tips and not explicit directions and you should always research your own solutions instead of thinking of these tips as rigid tutorial, step-by-step directions.

How to Approach Tough Programs

Our class can get quite frustrating, especially for students who don't have an extensive background in programming. The best approach to dealing with issues and errors in such cases is to start with very simple but fully functional code and then slowly adding changes to avoid an accumulation of errors.

The overall approach I might recommend is:

  1. List the functionality of what you want the code to do
  2. Outline, in pseudocode or a bulleted list format only, the structure of your solution
  3. Then, implement only the first one or two things and ensure it's functional and runs without errors after each small change

If you still run into issues, the next step would be to search for the specific error online to see if there are helpful posts on stackoverflow or other such sites on the Web. If that fails, as well, you can post in the discussion forums online with the following:

  1. List what you're trying to do
  2. Post the specific code and also what you expected it to do
  3. Give the full error stacktrace and also your guess on what might be going wrong

Module 1

Error with setting up Grove RGB LCD

As you run the python program in Module 1, you might get an import error like this:

Traceback (Most Recent Call Last): File "Home_Weather_Display.py", line 40, in from grovepi import * Import Error: no module named grovepi

The best thing to do when you get any error is to search google for that exact error. E.g., when you search for that error above ("Import Error: no module named grovepi"), the first link suggests what the issue is likely to be and how to fix it.

In this case, the issue is likely that you're using Python 3.x instead of Python 2.7.x. If you are already using Python 2.7, then the issue is likely as discussed here:

https://forum.dexterindustries.com/t/no-module-named-grovepi-solved/382

You can fix that error by running sudo python setup.py install in the Python folder as mentioned in the solution by Karan there.

Some additional errors

At this point, you might get an additional error about the inability to find grove_rgb_lcd; if you again search Google for that specific error, the first link will probably help you fix the issue here:

https://forum.dexterindustries.com/t/solved-importerror-no-module-named-grove-rgb-lcd/3736

You can then run sudo curl -kL dexterindustries.com/update_grovepi | bash which runs through a bunch of installs but might not fix the problem fully. For the final part of the solution, you'll likely have to copy the grove_rgb_lcd.py file over to the same location as the project (e.g., the Home_Weather_Display folder) as discussed here: https://forum.dexterindustries.com/t/home-weather-display-error-with-the-grovepi/4822/7 E.g., you might want to do something appropriate on your particular installation like sudo cp ~/Desktop/GrovePi/Software/Python/grove_rgb_lcd/grove_rgb_lcd.py ~/Desktop/GrovePi/Projects/Home_Weather_Display

Dealing with errors in general

As mentioned above, one of the most effective ways to deal with errors is to search for them on Google! E.g., if you get an IOError (like 5 or 110), a very good technique to troubleshoot these errors is to look for them on Google; e.g., you can search for actual parts of the error produced by putting those parts in quotes, as below:

"Home_Weather_Display.py" "IOError: 5" setRGB(0,255,0)

And the first link points to the IOError 5 with some great tips to fix it:

https://forum.dexterindustries.com/t/home-weather-display-rgb-error/6048/2

If you're getting IOErrors, you should likely also confirm your connection, as per the following which was in response to an IOError 110:

https://forum.dexterindustries.com/t/home-weather-display-error/3580

Module 2

Issues with Installing MQTT Mosquitto on Stretch vs Jessie Raspbian/Debian Distros

Reference: here is a great reference for installing the Mosquitto MQTT Broker on Pi with Websockets: https://xperimentia.com/2015/08/20/installing-mosquitto-mqtt-broker-on-raspberry-pi-with-websockets/

As you start to install Mosquitto, you might get some errors after typing this command:

sudo apt-get install mosquitto mosquitto-clients

The errors might be something like this:

The following packages have unmet dependencies:
mosquitto : Depends: libssl1.0.0 (>= 1.0.1) but it is not installable
Depends: libwebsockets3 (>= 1.2) but it is not installable
mosquitto-clients : Depends: libmosquitto1 (= 1.4.15-0mosquitto3~jessie1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

The issue is that your installation of Raspbian might update automatically from "Jessie" to "Stretch" which, from the forums, points to part of the problem. The repository for Jessie wouldn't pull the necessary dependencies so you'll have to manually get them and install them. You can confirm if you're on Stretch by checking your OS version:

cat /etc/os-release

If you are on Stretch, you can follow some of the tips in this article, especially the bits about checking the versions of installed packages under the installation heading:

https://www.eclipse.org/forums/index.php?t=msg&th=1089302&goto=1773991&

The main fix will be from a comment by sevos on 2017-10-11 in this thread:

https://github.com/eclipse/mosquitto/issues/529

This will likely require you to issue the following commands:

wget http://ftp.us.debian.org/debian/pool/main/o/openssl/libssl1.0.0_1.0.2l-1~bpo8+1_armhf.deb
wget http://ftp.us.debian.org/debian/pool/main/libw/libwebsockets/libwebsockets3_1.2.2-1_armhf.deb
sudo dpkg -i libssl1.0.0_1.0.2l-1~bpo8+1_armhf.deb
sudo dpkg -i libwebsockets3_1.2.2-1_armhf.deb

Finally, you can then issue this command:

sudo wget http://repo.mosquitto.org/debian/mosquitto-jessie.list (OPTIONAL)
sudo apt-get update (OPTIONAL)

sudo apt-get install mosquitto mosquitto-clients
sudo service mosquitto start

The package names are out of date so you might have to navigate to the parent folder in a web browser and find the closest match to the specified package, making sure to download the armhf architecture version and the appropriate libwebsockets8 package based on the instructions in the eclipse.org post, for example. After a successful installation, you should be able to start the service and run through the tests to verify the installation.

This might also result in some extra list files (e.g., if you tried the unsuccessful installations multiple times), you can remove those extra invalid files if they bother you by doing something like:

sudo rm mosquitto-jessie.list.1

change the number for as many installs you have (ex: 14)

the directory should be

/etc/apt/sources.list.d

The /etc/apt/sources.d is the location for configuring repositories for packages in "drop files". In other words, any file you drop into that directory is included in the configuration.

If it doesn't let you remove them, you may have to change to that directory to remove each one. To graphically see how many extra files you have, just click on the file icon at the top right, and then follow the directory path. each file will be labeled the same, but with a number on the end. any of the ...Jessie.list.X are copies of the actual file. you can delete all of those from the command line. As a worst case, you can do a

sudo rm mosquitto-jesse.list
sudo apt-get update

Mosquitto installation might not be needed on Stretch

Mosquitto is now also part of the standard Debian Stretch repos; if that's the case for you, you can skip these steps:

wget http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key
sudo apt-key add mosquitto-repo.gpg.key
rm mosquitto-repo.gpg.key
cd /etc/apt/sources.list.d/
#sudo wget http://repo.mosquitto.org/debian/mosquitto-jessie.list
sudo wget http://repo.mosquitto.org/debian/mosquitto-stretch.list

And only do these steps and it should be okay:

sudo apt-get update
sudo apt-get install mosquitto mosquitto-clients
sudo service mosquitto start

But, of course, your mileage may vary. :) Here's a link to a video for installing Mosquitto on Stretch using the standard repos:

https://www.youtube.com/watch?v=AsDHEDbyLfg

You should test your Mosquitto installation as per the following:

Example code is in the GrovePi/Software/Python directory wherever you installed the GrovePi software and also the mqtt_client_test.py code can be transferred to a copy of your weather display code and tweaked to write that data to mqtt.

JSON Parsing Values

You can use the following to parse out integer values:

msg.payload = JSON.parse(msg.payload.blue)
return msg;

Check your python version

Some of the sample code using paho-mqtt 1.2 defaults to Python 2.7 so if you're using Python 3.x, you might need to do a pip3 install paho-mqtt python-etcd

A similar project...

A similar project is the one you can find here: https://thingsboard.io/docs/samples/raspberry/temperature/

YouTube Intro

Here's a great intro to MQTT on Raspberry Pi on YouTube: https://www.youtube.com/watch?v=Pb3FLznsdwI

Module 3

Getting started with Node-Red

This link has a great tutorial on how to get started with Node-Red: https://randomnerdtutorials.com/getting-started-with-node-red-on-raspberry-pi/, including a simple installation approach using curl: bash <(curl -sL https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/update-nodejs-and-nodered)

And here's a video that takes you through updating your Raspbian and Node-RED: Node-Red Raspberry Pi Updating and Securing: https://www.youtube.com/watch?v=z9a_ztJqaII

References for using Node-RED

Node-RED's official API guide to writing functions: https://nodered.org/docs/writing-functions

Troubleshooting Node-Red Package Manager

The easiest way to install Node-RED is to use the node package manager, npm, that comes with Node.js. We'll use npm to install additional UI nodes to Node-RED in the next module. So you could, for example, use the GrovePi nodes for Node-RED (which are available here: https://flows.nodered.org/node/node-red-contrib-grovepi) to interact with our sensors but, instead, we'll create nodes that will interact with those sensors via MQTT so we can extend our approach later on to control the Raspberry Pi from remote and not just locally, as we would have to with the GrovePi nodes for Node-Red.

If you have issues with the nodered installation and especially the npm installation due to unmet dependenices, you can take the old Ubuntu-esque approach:

sudo apt-get remove nodejs npm ## remove existing nodejs and npm packages
sudo apt-get install curl  
curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get install -y nodejs

Or, if you want the latest approach, follow the approach at https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-debian-9

sudo apt update
sudo apt install curl
cd ~
curl -sL https://deb.nodesource.com/setup_10.x -o nodesource_setup.sh
sudo bash nodesource_setup.sh
sudo apt install nodejs
sudo apt install build-essential
You might need to add the -y option to install dependencies, as well.

Data Sources/JSON Function

This part of the directions might be confusing: "Since gauges take a single data point this will require you to define two more data sources, one for Temperature and one for Humidity. It will also require you to write at least one custom function (in Javascript, see the info in the function node and) to extract the data Writing Functions from the original Temperature and Humidity message payload. In total, you will be adding two function nodes and two data source nodes to the flow."

As with all things in computation (and physics), there are numerous ways to approach activities like this and if you've developed a solution that works, I'd consider that success. One approach to solution is writing the right custom function in JavaScript.

The need for the custom function was due to the gauge only recognizing a single data point and the datasource node recognizing the entire data field. In those instances, if you try to connect the json node directly to the datasource node and configure the datasource node to only take a certain data point, you might find that you either receive no output or only the first data point on your dashboard. In this case, flowing variables directly to the datasource node would also produce no output because it only recognizes a json object.

Using the custom function, however, you can take a single data point from the original json object and create a new object with only that data point as the message payload. Here's an example of what you might do for humidity, in this case:

var humMsg = {payload: msg.payload.data.humidity};
var tmStmp = {payload: msg.payload.timestamp};
return {
      "payload": {
              "timestamp": tmStmp,
              "data": humMsg
      }
};

In terms of converting temperature to celsius, you can do it on the python side or on the JavaScript side using something like this:

// Sample payload
// {"timestamp": 1519609945872, "data": {"range": 511, "degrees": 222.0, "temperature": 19.0, "humidity": 47.0}}
temperature = msg.payload.data.temperature
for (var key in msg.payload.data) {
    //if (key !== "temperature")
    delete msg.payload.data[key];
}
msg.payload.data.temperature_c = temperature.toFixed(1);
msg.payload.data.temperature_f = (temperature * 9 / 5 + 32).toFixed(1);
return msg;

You can do this for each sensor and connect a datasource node to each with the default setting for data field and get the expected output.

Another approach is to just add a separate data set in the dictionary (perhaps calling it 'displaydata' in your code). In that way, you would have:

'displaydata': 'timestamp', 'temp_hum', 'temp', 'hum', 'rotary_angle'
'temp_hum' would have two variables defined and 'rotary_angle' with four, whereas both 'temp' and 'hum' would only have one.

Then you can create the four datasources for the dashboard, connect them all to the json node, and then create the charts based on the appropriate datasource. If you approach it this way, you won't have to write a custom javascript code if you just reiterate the variables in defining the python object. The only redundancy here is that although you are not truly duplicating info given, that it is pulling temp and humidty from the same variable within the loop.

Troubleshooting Node-Red

If you're having issues with publishing your temperature/humidity to Node-Red, you might want to ensure everything is setup correctly. For starters, please check to see if you are able to subscribe to the topic using the terminal, e.g., using the mosquitto_sub -h command as shown below:

Second, please ensure that the topic title (the -t SCHOOL/COURSE/...) matches the title to which you are publishing in the MQTT node on Node-RED.

Edit the MQTT Input node called Raspberry Pi Sensor Data:

Edit the Debug node:

Alternative Node-Red Check

As an alternative to ensure your Node-RED is working, you could also build everything on Node-RED using the following nodes:

From input, add these two nodes:

From output, add these two nodes:

Connect the inject input node ("Test String Message") to the mqtt output node ("MQTT OUTPUT") as follows:

Connect the mqtt input node ("MQTT INPUT") to the debug output node ("msg.payload") as follows:

Then, hit "Deploy" and then hit the button as highlighted bellow

You should see your string message, Test, in the panel to the right:


Eventually, please ensure your JSON message payload is formatted properly in Python, as well:

#JSON
sensor_data = {
  'timestamp': 0,
  'data': {
    'temperature': 0,
    'humidity': 0
  }
}

sensor_data['timestamp'] = integer_milliseconds_since_epoch
sensor_data['data']['temperature'] = tempF
sensor_data['data']['humidity'] = hum

# Print the data locally:
print(json.dumps(sensor_data))

Module 4

Random NodeRed Issues

If everything in Node-RED works until you put the slider node in (i.e., it deploys but when you go to the Dashboard it comes up saying something like, "please add some UI nodes to your flow and redeploy", the issue might be to first update everything and then make sure you have not added the node-red-contrib-ui from the palette as it is no longer supported (you might have done this if you, for example, had the switch node from the contrib-ui nodes as your option and used it). Once you remove this, everything should work fine.

Slight typo in instructions: when you see the example of sending the form data in Postman as:

{
 blue: 255,
 red: 0,
 green: 124
}
it should actually have quotes around the strings, as in the pictures later on, unless you've defined them as variables:
{
 "blue": "255",
 "red": "0",
 "green": "124"
}

Similarly, when you setup the Blue LED Function Node in Node-Red, you'd have to add the quotes as follows:

msg.payload = {"blue": msg.payload};
return msg;

This is because the led_fadeMQTT.py uses that payload in the on_message() function as:

payload = json.loads(msg.payload)
# the legal values for analogWrite are 0-255
grovepi.analogWrite(BLUE_LED, payload['blue'])

This is similar to the direct MQTT messages published via mosquitto_pub -d -t "SCHOOL/CLASS/leds" -m "{\"blue\":0}", etc.

Tips for the REST API

If node-red doesn't seem to allow any form of printing, you can try to troubleshoot it using print, this.warn, this.log, etc.  If they all cause an error message, you can publish the contents of the debug object in the Debug panel. Also, please note that the msg.payload has two attributes like: "blue":"128". Since parseInt() is not able to convert "blue", it might return NaN (and become 0), as seen in the attached image:

This is thus similar to the JSON.parse() from Module 2 as you'll need something like msg.payload = parseInt(msg.payload.blue);

You could also do something like this, e.g.:

var blueString = msg.payload.blue;
msg.payload = parseInt(blueString);
return msg;

You might have to use the javascript parseInt() function on msg.payload.blue (blue is an element within the payload object) in order to pass through the values using postman and setup the REST API. You should then be able to setup the UI and GET LED State in the webpage. Please refer to the attached diagrams representing the different values after changing the LED intensity (within blue).

Nota bene: please be sure to click the checkbox next to the Key in Postman before clicking Send, as seen above. If for some reason you get 404 errors on raspberrypi:1880/leds, please do check if you can access raspberrypi:1880/ and, if not, try the IP address instead as it might not be reaching it on the network via dns lookup alone. If you can reach the / node (as opposed to the /leds), it's likely an issue with how the nodes were setup so I'd recommend getting rid of all the nodes and re-adding each one, step-by-step, and checking the "Send" after each addition.

Nota Bene: The pace of progress is quite fast in this field so the newer versions often change certain UI features. The good news is that you're not constrained to Postman; there are a host of other API testing alternatives and please do feel free to research and use any that you like, including some that are listed here:

https://alternativeto.net/software/postman/

In general, please do remember that these hints are intended to be little tips and not explicit directions and you should research your own solutions.

Module 4: Security for NodeRed (OPTIONAL)

I am sure most of you have now realized that by defualt there is no username and password setup to login to Node-Red. The following is how to setup basic username and password authentication into the Node-Red envirnoment. Adding SSL/TLS would provide encyption to better guard the login process and session data.

  1. Open a terminal and shutdown the Node red service. The command is node-red-stop.
  2. In the terminal and navigate to the "~/.node-red" directory.

    1

  3. Next will make a backup of the "settings.js" file, so that just in case a problem occurs, your original node-red settings will have a backup copy on file. Run the command cp settings.js settings.js.bak.

    2

  4. Now that we have a backup, we can edit the file. Before we edit the file, we will need to create a bcrypt password hash. This password hash will be used inside the settings.js file, when setting up the username & password.

    Start by running the following command in order to install the needed package: sudo npm install -g node-red-admin

    3

    Nota Bene: If you get an error, please check to see if npm (Node.JS Package Manager) is already installed and just needs to be updated.

  5. When the package is finished installing, we will generate a password hash. Using the node-red-admin tool, run the following command, and when prompted type in your desired password. I used the "testing" here, so no need to try to crack the bcrypt hash.

    node-red-admin hash-pw

    5

    With the password hash created, make sure that you copy and paste the password hash output on the screen to a temporary text file. This hash will be used next.

  6. With the password hash created, open the settings.js file with your text editor of choice, I will used vim.
  7. Scroll down the file until you find section labeled "Securing Node-Red".

    7

  8. Highlight the section in the "Securing Node-Red" section labeled "adminAuth" down to the end of the section. Copy this block of text.

    8

  9. Now create a few spaces below the section, and paste in the copied text. Remove the "//" in front of each line in the newly copied section.

    Where the text says "admin", replace this with your new username. In my case I used my first initial, middle initial and lastname. Make sure to leave the quotes around the username, as Node-Red is looking for a text string.

    Same for where the text has "$2a$08......" the default hash value, take the hash of the password that you created and saved to a text file in step 5 and paste it in place of the default one. Again remember to leave the quotes.

    9

  10. When you are done editing the file, save your work. With the new settings.js file created, start the node red service. The command is: node-red-start
  11. When your Node Red service is finished starting, open a web browser and go to your Node-Red server address. You should see a login page now. Provide the username and password you created. In my case this was username: ajcollins & password: testing.

    11

  12. Once you login you should see your Node-Red flows.

    12

  13. -- If it does not work --

    1) Turn off your node red service

    2) Copy the settings.js.bak file to the settings.js file to overwrite any previous changes. The command is: cp settings.js.bak settings.js

    3) Turn the node red service back on

Here is the link to the video that I found on how to set this up. Best of luck. https://www.youtube.com/watch?v=_cpqvUoR044

Some Helpful Videos

Some helpful video references for this week are:

Module 5

Install Meteor on a second PC

Meteor, or MeteorJS, is a free, open-source framework for building web and mobile apps in pure JavaScript. The general expectation is that you'll install Meteor on a different PC. The assumption is that most people have Windows PCs so using a package manager like Chocolatey is one of the easiest routes for installing Meteor. If needed, you could set it up as a VM or install on a non-Windows machine or even on the Pi but that would introduce more complexities. As such, the easiest route is to install it on a Windows PC using Chocolatey.

The best place to start is via https://www.meteor.com/install but if you wanted to install it on Windows, you can do so using either a elevated command prompt (one with Administrator privileges) or using Power Shell, as shown here: https://www.youtube.com/watch?v=OEIyk80XuDU

The default installation of Meteor also installs MongoDB in the meteor package in a directory named .meteor. While meteor is running, you can access meteor at http://localhost:3000 and the MongoDB database called meteor at mongodb://localhost:3001/meteor.

Meteor Simple-ToDos App Blaze Tutorial and GitHub

Also, if you're trying to follow the app, perhaps their github site where they commit each step of the tutorial separately might be helpful:

https://github.com/meteor/simple-todos

This github repo is based on the blaze tutorial (as they describe in the repo) linked in the Module: https://www.meteor.com/tutorials/blaze/creating-an-app

You might want to create the simple-todos app by passing the full parameter as that will create the imports directory along with the rest of the project for you. This might make the process a little smoother:

meteor create simple-todos --full

Here is a nice 48-minute tutorial that features a step-by-step process for creating a Meteor App: https://www.youtube.com/watch?v=9494-2E4riQ It actually builds a note-manager app as opposed to a to do list but the underlying ideas are the same and the video routinely references the Blaze Todo App Tutorial as well. Students have liked this tutorial in the past as it provides much more thorough explanations throughout the process.

Using diff on github

You can see differences for any step by doing the following:

# import the remote references
git fetch https://github.com/meteor/simple-todos.git refs/heads/*:refs/remotes/other/*
# get the commit hash for your step (replace 10.3 with your step number)
git log --no-merges remotes/other/master --grep="Step 10.3" --format="%H"
# view any differences- replace the commit hash with the value from the previous command,
# and add a file to the end to only see the differences for that file.
# git diff .. 
# this command will show you any differences between your tasks.js and the "Step 10.3" version of theirs,
# but you should replace the commit with whatever step you are on
git diff ..96a1a2893e1092ba0d36bf344e39ea00f7b90396 imports/api/tasks.js

on linux or unix, you should be able to combine the last two steps and just run:

git diff ..$(git log --no-merges remotes/other/master --grep="Step 10.3" --format="%H") imports/api/tasks.js

replacing the step number with your target

Tips for Meteor's Simple-ToDo Tutorial

The main link for the Simple-ToDo app tutorial with React (instead of Blaze) is here https://www.meteor.com/tutorials/react/creating-an-app

There is a very nice playlist of videos that takes you through the entire process of doing the default meteor Simple-ToDo tutorial in a series of videos here: https://www.youtube.com/watch?v=UsBN3RDy3Qw&list=PLhScwEnhQ-bmkhurUJJj16TMUKF0zLmxC

There is also a simpler tutorial for an older ToDo app in Meteor here: https://www.tutorialspoint.com/meteor/meteor_todo_app.htm as well as this tutorial for the ToDo app from a book extract: http://meteortips.com/second-meteor-tutorial/getting-started/

Issues with ToDo sample app

You might run into issues as the tutorial tends to get outdated quickly. Here are some links if things aren't working quite as well as the original tutorial: https://forums.meteor.com/t/basic-to-do-app-not-working/20796 and https://github.com/meteor/tutorials/issues/30 and https://forums.meteor.com/t/simple-todos-tutorial-gives-no-tasks-from-mongodb/22089/6

Alternative Meteor Video Links/Tutorials

The Simple-ToDo app tutorial is just a sample app using MeteorJS. If you find the Simple-ToDo app, and its associated tutorial, daunting or particularly troublesome, please do feel free to research your own tutorial on the web to built any Meteor app at all as our goal is to get familiar with Meteor apps and how to build and adapt them so we can build an IoT app based in Meteor in the next module.

With that in mind, you might want to reference this video tutorial on creating a similar (but different) app using Meteor: https://www.youtube.com/watch?v=9494-2E4riQ

The link in the module (https://youtu.be/LoHxiZUtg7s) is broken; here are some amended/substitute links; please do give them a try instead:

I've also been enjoying this one as it is closer in technology to what is shown in the meteor starter instructions.

https://www.leveluptutorials.com/tutorials/meteor-react-for-everyone/series-introduction

It incorporates React with Meteor as opposed to Blaze as that's what the latest Todo app uses.

Meteor Installation NPM and GIT issue

If you are working on the Meteor tutorial on Windows 10, you might run into some issues when you try to run it including the following error on the command prompt when trying to create the project.

Error: Error: Could not install npm dependencies for test-packages: Command failed:

I'd recommend running everything in an elevated command prompt with Administrator privileges but, if that doesn't work, you might have to install Git. It turns out that Git is a requirement that Meteor doesn't download and install for you. There is a lot of information in the error that points to this being the issue and it is also delineated on this forum post:

https://github.com/meteor/meteor/issues/8585

You can download and install Git from here:

https://git-scm.com/downloads

After downloading it, please reboot the computer and ensure you run the command prompt with elevated privileges to install Git and retry the Meteor installation.

Also, when you install Meteor for Windows, the version here might work better:

https://install.meteor.com/windows

You might also have to uninstall Meteor using Chocolatey, clean any lingering data from your system, and install using an older Windows installer. This was all suggested in a forum post from symptoms you might see after manually installing all of the missing npm packages. E.g., an error you might see the following error on variety calls in the cmd prompt:

EPERM: operation not permitted

If you do, please do check this post/instructions to get a working install.

https://github.com/meteor/meteor/issues/8139

Nota Bene: please install Chocolatey using the command prompt as an administrator before you install Meteor.

Nota Bene: the best order might be to install Chocolatey and GIT first and then Meteor.

Module 5 Meteor Installation NPM and GIT issue -- Alternative

You might run into alternative issues with the installation. If you do, another thing you can try is to delete all of your meteor and chocolatey files (by finding folders and deleting them manually) and start over. The missing GIT installation will no doubt be a persistent issue so you can install that by using these instructions:

https://www.develves.net/blogs/asd/articles/using-git-with-powershell-on-windows-10/#installing-git

Once this is done, you might still get an error about a missing "babel runtime npm package" and it will probably lead to the error, "exited with code 1" Your application is crashing...  but with the babel runtime missing. If you follow this error, it will likely want you to do a:

meteor npm install --save babel-runtime

meteor npm install -save @babel-runtime

But you might have to do this twice in order for it to work.

Module 5 Meteor Testing

You might run into issues while testing your Meteor application where the application itself seems to be running as expected with all the features working properly but the testing step at the end is producing errors in the command prompt and no output in the browser window.  The error might look like the following:

Error

It's a pretty lengthy list of errors that seems to be related to invoking the method 'mocha/runServerTests'.  The errors include:

"[object Object]" reporter blew up with error:

AssertionError [ERR_ASSERTION]: path must be a string

invalid reporter "[object Object]"

After you check for something wrong with the code, the version of meteor and mocha, or using a different testing method, you can reset the project and run the tests as an admin but if nothing still seems to work, it might have to do with with having duplicate code in one of your folders.

No matter what, though, you should install a different test package due to the upgraded version of meteor to get it to work correctly.  E.g., you can uninstall the practicalmeteor package and  install  "meteor add cultofcoders:mocha" and then run it as " meteor test --driver-package=cultofcoders:mocha".  This should hopefully solve the issues and the testing should work.  Here are two links regarding this.

https://github.com/practicalmeteor/meteor-mocha/issues/100

https://blog.meteor.com/announcing-meteor-1-6-1-50aad71da4e6

I think removing practicalmeteor might be the key.  I've also periodically run into EPERM: operation not permitted errors when attempting tests.  I had originally gotten around this by running the command prompt as an admin but running them as an admin might be dangerous so I found that running "reset meteor" fixed the issue, although it deleted all of my tasks.

An alternative option to the cultofcoders.mocha is to upload meteortesting.mocha also.   The cultofcoders.mocha states that it is the practical meteor made to work with meteor 1.6.0 and newer coffee versions. Please see here for the meteortesting.mocha links:

https://forums.meteor.com/t/meteor-1-6-1-practical-mocha-invalid-reporter-object-object/41911 

Some potential source code changes

Some of the important changes are: ui/body.html: create a form to add new tasks ui/body.js: import ui/task.js here ui/task.js: add event handlers import { Template } from 'meteor/templating'; import { Tasks } from '../api/tasks.js'; import './task.html'; Template.task.events({ 'click .toggle-checked'() { // Set the checked property to the opposite of its current value Tasks.update(this._id, { $set: { checked: !this.checked }, }); }, 'click .delete'() { Tasks.remove(this._id); }, }); ui/body.html: <body> <div class="container"> <header> <h1>Todo List ({{incompleteCount}})</h1> <label class="hide-completed"> <input type="checkbox" /> Hide Completed Tasks </label> <form class="new-task"> <input type="text" name="text" placeholder="type to add new" /> </form> </header> <ul> {{#each tasks}} {{> task}} {{/each}} </ul> </div> </body> body.js: ... Template.body.events({ 'submit .new-task'(event) { // Prevent browser form submit event.preventDefault(); // Get value from form element const target = event.target; const text = target.text.value; // Insert a task into the collection Tasks.insert({ text, createdAt: new Date(), // current time owner: Meteor.userId(), username: Meteor.user().username, }); // Clear form target.text.value = ''; }, 'change .hide-completed input'(event, instance) { instance.state.set('hideCompleted', event.target.checked); } You will also need to update the ui/task.html, ui/task.js, and api/tasks.js files to add the privacy button and selectively publish tasks. In the api/tasks.js file, you can also took the extra step of modifying the line: if (task.private && task.owner !== Meteor.userId()) {} You can remove the task.private component, creating the following line: if (task.owner !== Meteor.userId()) {} This condition exists in two places in the file. One is for the task of removing a task from the list, the other is for setting a task as checked. The original code set the condition that if a task is private and the current user is not the task owner (indicated by the !== operator), the program will throw a ‘not authorized’ error. The && operator indicates that both of these conditions must be met to throw the error. Because the previous steps installed code that would prevent a user from seeing another user’s private tasks, any task that a user can see will not meet both of these conditions because the task either belongs to the user or the task is not private. By removing the private requirement, the program will throw the error if the task does not belong to the user. You can make this modification in both places, resulting in a user not being able to remove a task or check a task as complete if the task does not belong to that user. tasks.js: ... 'tasks.remove'(taskId) { check(taskId, String); const task = Tasks.findOne(taskId); if (task.owner !== Meteor.userId()) { // If the task is private, make sure only the owner can delete it throw new Meteor.Error('not-authorized'); } Tasks.remove(taskId); }, 'tasks.setChecked'(taskId, setChecked) { check(taskId, String); check(setChecked, Boolean); const task = Tasks.findOne(taskId); if (task.owner !== Meteor.userId()) { // If the task is private, make sure only the owner can check it off throw new Meteor.Error('not-authorized'); } Tasks.update(taskId, {$set: { checked: setChecked } }); }, 'tasks.setPrivate'(taskId, setToPrivate){ check(taskId, String); check(setToPrivate, Boolean); const task = Tasks.findOne(taskId); // Make sure only the task owner can make a task private if (task.owner !== Meteor.userId()) { throw new Meteor.Error('not-authorized'); } Tasks.update(taskId, { $set: { private: setToPrivate } }); }

JavaScript Debugger

The javascript debugger is very helpful in figuring out what JS applications are doing and since Meteor is new to all of us, I thought it might be helpful to review the JS Debugger for the ToDo app, especially since we don't have any references for it in Meteor's ToDo-App Tutorial. The debugger can help us see what's going on under the hood and how to troubleshoot when an issue comes up. It can be time-consuming and quite frustrating to directly pore through the framework code to try to figure out any issues that arise and the JS Debugger can help with that quite a lot.

You might have to research how to do this in other browsers, but for chrome it is pretty straight-forward. Open up chrome's dev-tools by right-clicking and selecting "inspect" or you can go straight to the javascript console using the command keys from the answer here: https://stackoverflow.com/questions/66420/how-do-you-launch-the-javascript-debugger-in-google-chrome.

inspector

Explore the tabs at the top of the inspector window if you haven't before, there are a lot of useful tools there. I use the console tab for trying out Javascript. With the inspector tools open, add a debugger; line wherever you want in your client-side code.

debugger

When you refresh your app (Meteor should for you) you will be dropped into the debugger. There is a bunch of useful information about your local scope in the sources tab.

sources

If you go into the console tab from there, you can execute/try out different lines of javascript and see the results without changing your app code. The autocomplete is really helpful for seeing which methods are available to you.

autocomplete

fetch

From there you can try different methods to get what you want from that point in the code's execution. Once you've figured it out, you can add what you learned back into your real code and remove the debugger line. Even with Meteor's auto-refresh, this saves me a lot of time and I'd highly recommend it. Note that you might find some interesting things, like your debugger point getting hit more than once. Some methods might be called more than once during page load, and the scope might change a little between calls. Usually, the first time I add a debugger I click the resume button in the "Paused in debugger" window at the top and see if the page loads entirely before hitting the debugger again, just to get an idea of how often that method is called and what to look out for.

Module 6

MongoDB Introduction

MongoDB in 30 minutes: https://www.youtube.com/watch?v=pWbMrx5rVBE

Issues with Node-Red and MongoDB

There can be a bunch of different issues with MongoDB and Node-Red; if you find that Node-Red won't connect with MongoDB, one technique that seems to work with some of the tougher to track down errors is to reinstall Node-Red.

Some questions to ask if things don't seem to be working are:

You can check to see if the tasks are in the mongo db by seeing what db.tasks.find() shows (you can use this after doing meteor mongo to see what's in a specific Collection). Does that show anything? We can also add tasks directly (using db.tasks.insert()) to see if there's an issue with the db or not.


You can also use external programs like the MongoDB Compass GUI to connect to the MongoDB server to see if your strings are being saved in there when you add items using something like Tasks.insert({text: "Add new todo", createdAt: new Date()}) in the JS console in Chrome or in the meteor shell. You do need to ensure the JS has a return statement: https://stackoverflow.com/questions/31353976/meteor-changes-in-collections-are-not-reflected-in-rendered-html

UUID Issues

If you have issues concatenating the uuid, please save it to a variable first instead of using the function directly in the concatenation. I.e., you might want to try something like this:

uuid = uuidgen.generateUuid()
topic = "SCHOOL/CLASS/sensor/data/"+uuid

instead of

topic = "SCHOOL/CLASS/sensor/data/"+uuidgen.generateUuid()

Finally, to follow up on the issues mentioned in the announcement post during week six about being unable to connect. To troubleshoot, I ran an older project that connected to 'test.mosquitto.org' and it connected with no issue. After some playing around, I found the fix if anyone is still experiencing issues. In your mqtt input node, ensure you enter your topic , i.e. 'SCHOOL/CLASS/sensor/data/#'. Most importantly, check the mqtt output node. The server should be set to test.mosquitto.org (I set the port to 1883), and the topic field should be left empty. If you enter your topic here as well, it will cause connection issues.

This can look like:

local_client.publish('SCHOOL/CLASS/sensor/data/' + uuid, json.dumps(publish_data))

In addition to saving the UUID as part of the topic as indicated by the instructions, you can also add the new UUID variable we created above to the JSON string that contains your sensor data, something like:

publish_data = {

"temperature": temp,

"humidity": hum,

"uuid": uuid

}

This way, the UUID is published as part of the object containing the sensor data and can be extracted with a function in Node Red.

DEBUGGING: This is also the way to pinpoint the UUID issue: connect a debugging node to the incoming MQTT node will show the UUID in the topic, which could indicate whether the outgoing MQTT node was removing it when forwarding the message to the data broker.

Module 7

Issues

At the beginning of Lab 7, you have to create a new meteor project with both default settings and using the '--minimal' flag, add the required packages, remove insecure and autopublish. Next, you have to delete all of the boilerplate files leaving only the .meteor folder and the package.json file.

You have to edit the package.json file (more about package.json) to remove the following code, including the comma:

,

"meteor": {
  "mainModule": {
    "client": "client/main.js",
    "server": "server/main.js"
  },
  "testModule": "test/main.js"
}

This should leave something like the following:

{
"name": "iot",
"private": true,
"scripts": {
"start": "meteor run",
"test": "meteor test --once --driver-package meteortesting:mocha",
"test-app": "TEST_WATCH=1 meteor test --full-app --driver-package meteortesting:mocha",
"visualize": "meteor --production --extra-packages bundle-visualizer"
},
"dependencies": {
"@babel/runtime": "^7.0.0-beta.51",
"meteor-node-stubs": "^0.4.1"

}
}

You might also have to remove your test directory and rebuild. This might also involve deleting the node_modules and reintalling them using the meteor npm install command.

Optional: this might help to call the correct file name in the package.json file. You still might have to rebuild the node_module folder with the error of "meteor npm install --save @babel/runtime."  Once that is done and the package is calling router.js instead of main.js, it should run (but this whole step might not be needed for you).

You should eventually see something like:

dash

Here's a screenshot of what you might get from the Node-Red JSON node in debug. This is before the function node that either adds everything to the collection or overwrites the data in place.

node

Here is a potential node setup, as well. This doesn't have both function nodes connected so you could verify the separate behaviors.

setup

Please note that this is an Object from the MQTT input and not a string. 

Question about Multiple Collections

At the end, you need to add a new collection like sensorhistory in addition to the sensor collection. As such, you need to add another collection called sensorhistory and also edit your nodered flow. You can add a second function to add the timestamp collection in the collections.js file within the IoT app to include a new collection as well as define a new side nav for a separate dashboard in the single MongoDB.

So you'll have to add a second collection with a second tab for sensor history. Your node-red functions might be something like:

 

add sensor data attributes:

msg.collection = "sensors";
msg.topicParts = msg.topic.split('/');
msg._id = msg.topicParts[msg.topicParts.length-1];
return msg;

add sensorHistory:

msg.collection = "sensorHistory";
return msg;

Both nodes are taking data from Json function and then pushing to the mongodb output node.You have to create/update files in the iot application. You also have to ensure your application is not just subscribed to the sensors collection otherwise you will get the same output in both tabs despite publishing data to the sensorHistory collection. You should declare a template, dashboard, and helpers for the second collection. This is basically a matter of replicating the dashboard.html file and the sensor-table.js file and replacing sensorTable with sensorHistory.  You can update the sensor-table.html file with the same process, duplicating the sensorTable template and replacing with sensorHistory.  Here's a sample of the code:

 

*new dashboard:

<template name="sensorhistory">
    <h1 class="page-header">Sensor History</h1>
    {> sensorHistory}
</template>
 
*new helpers:
 
Template.sensorHistory.helpers({
    sensors: function () {
        return SensorsHistory.find();
    }
});
 
*new template:
 
<template name="sensorHistory">
    <div class="table-responsive">
        <table class="table table-striped">
            <thead>
                <tr>
                    <th>_id</th>
                    <th>timestamp</th>
                    <th>datetime</th>
                    <th>temperature</th>
                    <th>humidity</th>
                </tr>
            </thead>
            <tbody>
                {#each sensors}
                {> sensorRow}
                {/each}
            </tbody>
        </table>
    </div>
</template>
 

MongoDB and Node-RED Issues

If you get an "Authentication failed" error when you try to send data from Node-RED to MongoDB but can access and insert documents into MongoDB (e.g., in the meteor db or collection tasks via the mongo shell), it might be a username issue. This is especially true if you installed the Robo 3T GUI with MongoDB. One fix might be to reinstall Node.JS, uninstall Robo 3T, and then create a new username and password directly to Mongo through the command line. If Node-RED is crashing, as well, after a few posts to the db, one fix might be to disconnect the database and confirm you can open Node-RED safely, then delete all your flows and create entirely new flows.

Module 7: Error: Unexpected token u in JSON at position 0

This kind of error is usually the result of trying to do a JSON.parse(undefined). It could also be because the temperature sensor is sending an empty value. Either way, the result is that msg.payload is undefined .

In general, the best way to debug in NodeRed is to put in a catch code that's wired to a debug node for every tab.

Overview for Windoze

The goal of the lab itself is to create a new visual dashboard app which displays and stores real time as well as shows historical data. To accomplish this, you will first create a new Meteor project called iot and load it with a set of packages which includes packages such as bootstrap and router and basic login.

Here is a list of files:

At this point, you have to create a login and uncover the sideNav component:

You will then create a new Node-RED which will subscribe to the test.mosquitto.org topic from module six. You will add a JSON node to convert the JSON string to a JSON object:

After you deploy it and start to collect data from the sensors on the app, you can modify the Add Sensor collection function:

msg.collection = "sensors";

msg.topicParts = msg.topic.split('/');

msg._id = msg.topicParts[msg.topicParts.length - 1];

return msg;

Next, you can clear the mongo db on mongodb:

To put it all together, you can add another function to display the continues growth results at top of the in-place section:

You can review some of the MongoDB commands here: https://www.youtube.com/watch?v=pWbMrx5rVBE

Finally, you'll need to edit the package.json file to ignore these modules and run meteor npm install --save @babel/runtime so the package is calling router.js instead of main.js.

New package.jason code:

{
"name": "iot",
"private": true,
"scripts": {
"start": "meteor run",
"test": "meteor test --once --driver-package meteortesting:mocha",
"test-app": "TEST_WATCH=1 meteor test --full-app --driver-package meteortesting:mocha",
"visualize": "meteor --production --extra-packages bundle-visualizer"
},
"dependencies": {
"@babel/runtime": "^7.0.0-beta.51",
"meteor-node-stubs": "^0.4.1"

}
}

If you have a collection misconfiguration, where you use the same subscription topic arguments for both collections, this can result in the app not reading the data from your sensorHistory DB. You can fix this by fixing the collection arguments as found on https://docs.meteor.com/api/pubsub.html#Meteor-subscribe:

Remote MQTT Message Broker Unreachable/Connection Refused

If you find the test.mosquitto.org MQTT message broker is unreachable/sending connection refused, the most likely cause is due to a hosting issue at test.mosquitto.org.
If you experience the above issue, you can use an alternate remote MQTT message broker. Here is a list of Public MQTT Brokers: https://github.com/mqtt/mqtt.github.io/wiki/public_brokers

Module 8

Broken Discussion Link

Please do try this instead:

https://web.archive.org/web/20170708033524/http://www.time-management-guide.com/time-management-skills.html

Potential UnicodeDecodeError

If you get an error about UnicodeDecodeError when you run your python script, it might be an issue with the uuidgen.py file. You might be able to resolve it by replacing the uuidgen.py file with the same one we used for the Meteor application. Please do also ensure he four files for this project are in their own directory, then erase the new uuidgen.py file, then just copy and paste the older one into the directory.

The issue is likely related to the following line:

mac = netifaces.ifaddresses('eth0')[netifaces.AF_LINK][0]['addr']

It seems to need the encode() method from module six at the end of it:

mac = netifaces.ifaddresses('eth0')[netifaces.AF_LINK][0]['addr'].encode ('utf-8')

If you use the provided code with the above change, it should work. However, completely replacing the file could end up causing some minor issues with your Node-Red flow.