littlelogs

Keep a social journal of your work progress as you make and learn things.

May 2016

loudmouthman

#Particle Been away too long so lets be brief; Deaths in the family. new project take ons. travel and more planning. Oh and I’ll be in Belfast in June.

Meanwhile ; I was dickering around with a NFC/Arduino build and its grown into something else; something internetty. In looking to the question about arduinos and the internet I discovered Particle Photons ( formely Sparkfun ) and by golly they are a wonderful little development and playing wth board.

February 2016

loudmouthman

#Sales
When is the best time for hardware to die ? when you are leaving for a few days ! thats when ! So today we play hunt the suitable VDSL modem whilst awaiting the ISP to deliver a modem as a replacement. Life without the internet is so 1989 !

loudmouthman

#memcache
A recent issue for me was determining the speed of a curl response on a fairly busy website what I wanted was a way of tracking the current hours curl requests and discovering how many requests were made and how long they were taking. I could have pushed that data into a csv file but that would take up cpu time to push and write files; which eats more resources. To my aid came memcache which allows me to store a value in a scratchy area of memory which I can call back to. So problem solved; next problem .. how to drastically reduce uneccesary calls to curl and across the network. Heres a paste.bin

loudmouthman

#running As a side note; 20 miles of running this week with 6 miles of it today; and what I came away from after my run was a sudden realization of how I might improve the code I was previously writing. What I dont have is the time to apply that understanding.

loudmouthman

#Codeigniter #aws-sdk A few days of working with Codeigniter reaffirm that many developers working in codeigniter use frameworks to speed development and ignore well factored code. One thing of note is getting the sparks aws-sdk working to put an object into S3. it took me a while to work it out but the essence has been

$s3Obj=array('acl'=>AmazonS3::ACL_PUBLIC,'fileUpload'=> (  $imgPath   ) );
$s3->create_object($amazon_settings->aws_bucket_name,  $amazon_settings->aws_bucket_path .   $abs_imgpath    ,$s3Obj);
// imgPath was: /var/www/codeigniter/media/template/base_template/header.jpg
// aws_bucket_name : is the name of the bucket 
// aws_bucket_path  and imagepath are the full path from the bucket root to the file.; e.g.
'release/media/template/base_template/header.jpg

That wasnt actually documented anywhere in the library from sparks .

loudmouthman

#bash

I probably get around to using this one liner more than a few times in a month; Im usually needing to discover which IP address is abusing my log files the most

cat access_targetphrase.log  | awk {'print $1'} | sort | uniq -c | sort -n | tail -n10

This always reminds me that

1) Awk is superpowerful and constantly underused 2) Windows Log files are not nearly as easy to throw around as Unix ones. 3) I like making numbered lists.

loudmouthman

#running Kicking off today with a 14 mile run in the morning because I knew if I didnt do it at the start I was going to put it off in the day. So that is 34 miles of running for this week done. I find there are days when running doesn’t give me answers it just gives me a break. I cant answer emails, browse the web, look at alerts. All I can do is listen to my stride and the sounds of the world and pay attention to the rhythm . So if its 30 minutes of 3 miles or an hour of 7 mile or two hour of 14 its a break. Of course my legs may not agree with my regime so I take them on the bike and down to my other daily work place to answer emails and update calendars.

loudmouthman

#pineapple Another night of finding place for the pineapple, what I wanted was something to act like an automated Wireshark so that i could run up tcpdump with a collection of filters and then test away and come back and check the report to see if anything was in the clear. What i have is sort of like that except wireshark runs on my laptop. Not quite what I am looking for but its progress

loudmouthman

I am constantly unimpressed at the technical competency of some network engineers involved in either telecoms or photocopier installations on my clients network. It seems a constant battle to remind them that DHCP Server services cannot be enabled from their product when on the same network the main server is handing out IP addresses. They always insist that their product is super special sauce material which is why it HAS to be enabled. It never does. Anyway why after more than a decade of Cat5 am I still having his discussion. #networking

loudmouthman

Sometimes I open a BASH terminal and then completely forget what I was about to type. In this case yet another collection of bash scripts to hack together.

04 Feb 2016

2 Heart 0 comments

loudmouthman

#pineapplenano back to the pentesting corner again and this time with the pineapple nano and wireshark enabled as I take time to listen for packets exchanged and search out those which are boring old http. Looking for holes in a mobile app so I can give them the okay to move forward. So far the usual suspects are advertising agencies whose service seems to always be on http.

loudmouthman

#Windows10 #scratch Okay I admit it Windows10 has sort of won me over as an OS; sure im not as happy as a clam as I am in a bash shell but the robustness of this operating system in older laptops is leaving me looking like a miracle worker on resurrecting those old machines. So with that out of the way it was back to CodeClub. todays challenge for the kids was to build a simple towers of hanoi puzzle; this should keep them busy for the week.

January 2016

loudmouthman

#keystonejs

So I got the keystone cms working thanks to nvm and yo generator; the biggest security aaaaargh so far is the home page when created just publishes the default username/password of the admin owner. So glad that wasnt published to a more public port. Next stop is to work out why Keystone is so important because the content updates look like they will be a night of the mares.

loudmouthman

#NodejS #MongDB A new amazon instance for yet another agency. This time their special sauce is Nodejs/MongoDB and Keystone CMS. I suspect this means that unless they are using Git via sftp they will request shell access. Overall my concerns in using keystone are the lack of eyeballs on the product compared to Drupal/Wordpress but that is not a great reason to avoid it; mostly im looking down the thoughts on how to monitor ( read logs ) and observer ( check DB updates ) the site in such a way to be aware of attempts on the site ongoing.

loudmouthman

#bulletjournal My task manager preference remains Bullet Journal; it is a pen and paper list manager which I have been running for for two years now. I can flick through my journal and see over two years of tasks, projects, activites, notes and spending habits. As it is a Sunday this means I am down in the local cafe sipping a coffee and updating my Bulletjournal for the month ahead. Many systems have come and gone and yet this simple booklet of activity remains as ever my preferred route to routine.

loudmouthman

#Conference ; two days up in London enjoying some Monkigras talks. Catching up on old networking contacts discovering that some of Hack Horshams ideas are worth delivering on and revaluating the purpose of Nodered in terms of what we might deliver to teachers.

Speaking of teachers; the local HackHorsham teachers coder evening was a success so now we have more thing to talk to schools about bringing code to the class .

loudmouthman

#AWS #PHP #BASH
Its not event 10:20 and this morning i have edited a legacy php system to include an extra column of data which apparently has never appeared on a export and has only just been discovered. Ive recovered a lost xls file for a client because they are sloppy clickers and had dragged it to another share. I have configured a jailed user account with their own secure key and access to a folder on the filesystem so they can upload files directly. I have checked in on a STEM event I am helping deliver this evening and I have not even had my second cup of coffee.

loudmouthman

#RaspberryPi ; the Manic Minion punching game is assembled and on their bases ready for the kids to carry on coding . Based on a project suggestion from the team we have spent a few weeks discussing build materials, electronic components, and game interface ideas. They should be proud of their creation in this

loudmouthman

#AWS #Wordpress there are many schools of thought on automation of tasks but the reality is when I have to do a new wordpress install on a dev server it takes me all of 10 minutes and the act of creating apache configs and wordpress installations means I keep an eye on what is still running or utilised and ask the question . do we need it. in over two decades of automation so many tools have come and gone and few have replaced the mark one eyeball and attention to the installation directories.

loudmouthman

#googledocs So this little function in my google docs spreadsheet

=importData("http://targetdomain.com/secretypath/stats.csv")  

delivered the final twist to my automation of publication of cloudmetric stats which allows the client to assemble data from the upcoming event in a format which makes more sense

loudmouthman

Some Sunday Night Scratch something to bring to the local CodeClub ; an example of rolling dice to get a number. Something Random with just a hint of a problem in the animation; but it is an example of procedural code; as usual I set the challenge for the club kids to try and recreate it before looking inside.

24 Jan 2016

1 Heart 0 comments

loudmouthman

#MUTT wow been a long time since I read the mail spool at the command line but there i was using mutt to check in on the cron job outputs for the scripts. yep some errors to pick up on.

loudmouthman

#AWS #Cloudwatch #Bash ; For those interested in the relevant AWS command line content the link is included. Things I discovered is that you need to ensue that your source machine running the instance requests runs ntpdate to ensure its internal clock is consistently uptodate. I had too many problems with the times otherwise. I have another script on the remote EC2 webinstance which can pick up the logs and look over an hour of content and return the lines and bytes requested. all Written to csv for now.

loudmouthman

#AWS #BASH , and it is working … now to leave it a few hours and see if its correct. The collection of basic Cloudwatch metrics for a number of instances in addition to which a script which gathers the count of lines in the log file and the sum of bytes requested for that instance. I am hoping that the Loadbalancer requests will be far less than the Apache Log Lines which then informs me of the value of my varnish caches. I may also grab some varnish cache stats.

loudmouthman

#AWS #bash nothing like a good distraction and today it was getting AWS Command line working so that I could get the metrics for our instances and report back every hour.Sure here are other services that offer this but I dont want to add more external pressure to an internal routine. this way reduces the amount of open things into the backend. more to post on this later once I have it wrapped and operational

loudmouthman

#Apache Todays lesson to be taught to developers is that you REALLy do not want to mix and match http and https when you expect https to work; that being said its been a day of wondering just where in the stack something is caching old copies of a JS file and delivering them to the browser. Personally im adding custom headers into a stack and see what comes out the other end. stay tuned for VCL recipe mixes.

loudmouthman

#RaspberryPi #python The Raspberry Pi challenge project is all coming together the code work and the electronics pick up the sensor changes. Whats next is to get the kids identifying how to change assets in the game and how they are linked to files on the filesystem. I think this might need some video tutorials we can link to or at least something where the kids can spend longer and arrive with an actual structured lesson to work against. now.

loudmouthman

#Varnish #AWS Tonights question how to ensure that urls which did not start www or requests for port http:80 were redirected to the respective https://www address in varnish thereby saving a trip to apache where it would waste network and processor time. It turns out this is not too hard to deliver on providing your AWS loadbalancer is pointing to port 80 on the varnish instance; details of the edits in the PasteBin

loudmouthman

#AWS the loadbalancers on Amazon WebServices require you to pass the Public Certificate ( the file you got back from the supplier/registrar ) the Private Certificate/key ( the file you created with openssl options ) and you optionally ( hint not really ) the Ceritifcate chain. In this case It turns out that the Registrar may provide you their certificate chain and posting that into the final box means you have an SSL endpoint at the loadbalancer which handles the SSL between your clients and your apache instance. You can now configure the listeners to listen on port 443 out the outside and pass them to port 80 ( or whatever you need ) on the inside. Okay the communication between Loadbalancer and AWS instance is unencrypted in its journey but youve passed the network cost upstream of your app. It works well enough to allow me to run Varnish behind the loadbalancers and they in turn load a couple of instances.

loudmouthman

Speaking of Debugging … in working with #scratch for various codeclub experiences I have discovered that we are not teaching kids to code; we are teaching them to assemble bricks. What they do not learn in Scratch is debugging, is stepping through, is functional/procedural coding. Instead they assemble things where code can begin anywhere and the ability to add break conditions or logging is non existent. This is not teaching coding its teaching how to use an App. go on shout at me now !

loudmouthman

#wordpress #php in the land of debugging the log file is king. In my case the ability to add a little write_log function to other peoples themes just to see the step through of code in invaluable. I miss the days of the visual studio with its tidy IDE and its ability to step through code and add breakpoints and watches. In the world of php and coding on remote servers the log file is where I export my findings. I cant remember where I sourced the original code ( sorry ) but it goes into functions.php of the target theme.

if (!function_exists('write_log')) {
    function write_log ( $log )  {
        if ( defined(WP_DEBUG) ) {
        if( WP_DEBUG=== true){
            if ( is_array( $log ) || is_object( $log ) ) {
                error_log( print_r( $log, true ) );
            } else {
                error_log( $log );
            }
            }
        }
    }
}


// now you can call writelog('write something to the debug.log');  and all is well 
loudmouthman

#python #raspberrypi #pygamezero

So Pygame Zero exists to redress the imbalance in teaching python at school level and allowing code clubs to accelerate access to game coding ideas in classes. So what do I want to do when working through pygame zero documentation and not wanting to be out in my cold office. I forward Xwindows to my OSX machine and keep a terminal open for coding in python. I wonder if there are any nice shell IDEs for Python … who am I kidding its VIm or nothing for me.

loudmouthman

#ssh #Xwindows Some old school X forwarding going on; running up some python pygame and scratch examples on a remote Raspberry Pi which means I am using the X Forward command in my local .ssh config file; and ensuring that XForwarding is yes in the ssh/sshd_Config file on the server

host raspberrypi ForwardX11 yes

I logged in from my OSx machine and ssh’d over to raspberrypi then launched scratch at the commandline which invoked X to run remotely and send me the output for the window. Sure I could use VNC to get to my Pi but this way is more tcp robust.

in my .ssh/config

loudmouthman

Installing Ruby on Raspberry Pi so that I can install wpscan on Raspberry Pi so I can set up … actually you know what I might just make a Pi Cluster for the hell of it. But thats another day. Anyway ; go ahead with the basics …

apt-get install ruby-dev screen

then go ahead and run screen and fetch the following Ruby Install Bash Script

All sorted. great then grab the git
git clone https://github.com/wpscanteam/wpscan.git

and change into the wpscan folder ; nearly there .. then run up the gem bundling fun over on

sudo gem install bundler && bundle install –without test development

Then be patient again … Im noodling on the idea of Raspberry Pi scanners for dev sites

More on running wpscan in other logs.

#ruby #bash #wpscan

loudmouthman

#RaspberryPi #Electronics . So the thing about the Pi is that you cant tackle analogue content well on it. You have to sort of fudge it with a touch of timings and capacitors. There might be better answers but cant seem to make the rctime function in the event that i use multiple diodes on the board; its taken me a while to realise I could just put a larger resistor at the end of the line and this will allow me to run multiple inputs from the capacitors without worrying about the potential creeping back into each capacitor and creating inaccurate timings.

Long story short: I used a 3.9k resistor after the 1microfarad capacitor for the circuit and then shared then built the circuit to work with 4 pins in.

I based the idea from this article over on Raspberrypi-spy(http://raspberrypi-spy.co.uk/2012/08/reading-analogue-sensors-with-one-gpio-pin/)

Initial Design

loudmouthman

#Varnish ; every now and then the apache instance takes a little longer to kill sessions no longer in use and we end up running out of useable threads to deliver content over. Ahead of Apache( and php ) the varnish stack provides a useful cache element for objects delivered but from time to time it runs ahead of apache and we end up with 503s’ I’d like to log some of that activity but for now I want to tell varnish to be a little more graceful when talking to the backend; heres what worked so far in terms of letting objects live beyond their TTL in the event apache is being slow.

 # in the vcl_recv section add
set req.grace = 5m;

if (req.backend.healthy) {
                 set req.grace = 30s;
                } else {
                set req.grace = 1h;
                 }

Now to add some configuration items to tell Varnish to log those elements which meet criteria of fail.

loudmouthman

#AWS I do quite a bit of work with Amazon Web Services and this morning has mostly involved reviewing if the current deployment of services from AWS in China is suitable for us to deliver content currently hosted in Singapore. The problem to be resolved is that we really need a way for DNS to handle Zone information based on the origin request of the client. Anything in China needs to be pointed to china.

Chinanetcloud proves bloody helpful here to a degree but the reality is that I think this new few months will see me possibly building out a DNS server with BIND 9.10 it has the features I think we will need what I wonder is how to robustly handle the queries. so thats on my reading list for February

https://kb.isc.org/article/AA-01149/0/Using-the-GeoIP-Features-in-BIND-9.10.html

Older