Efficient image processing in Android - Placeholder

Different ways to do image processing in Android [ Read more ]

Adding attention mask in MultiHeadAttention layer of transformer using Keras and Tensorflow

[ Read more ]

Layout LM explained - paper reading

Document AI refers to techniques for automatically reading, understanding, and analyzing business documents or visual documents in general. Visual documents are electronic documents like a reciept or pdf document or photograph of an ID card. In this article I will be sharing my notes on the paper - 'LayoutLM: Pre-training of Text and Layout for Document Image Understanding' by Yiheng Xu et. al. When the paper was published the authors claimed this to be SOTA on multiple Visual Document Understanding tasks. [ Read more ]

Denoising algorithms in image processing - A comparision (part 1)

[ Read more ]

How to write Image to ImageWriter in Android

Android framework provides ImageWriter class as a way for applications to produce Image data int a Surface which can then be consumed by other components like CameraDevice. I found a couple of hiccups while trying to write jpeg images to ImageWriter buffers and found that there is scarce documentation around this out in the wild. In this article I'll share briefly on how to write jpeg images or YUV images to ImageWriter in Android. [ Read more ]

How to read an image file in C++ in Android with NDK

Software development in Android can be done using the Java SDK or Native Development Kit aka NDK made available by Android Open Source Project (AOSP). NDK is often used for writing high performance code like image processing algorithms. There may be times when you would want to load image from disk.

In this article I'll cover how to read an image file in C++ with NDK and JNI. The approach can in general be used to load any file in native layer. [ Read more ]

Hello World in Open GL

Learning OpenGL has been a long time todo and I have gladly taken a step 1. But I realised it's a lot of steps to just to get to hello world application or in this case I must say - 'Hello Traiangle' application. In the past writing a blog article has helped me get a better grasp of things, have a easy to access notes and maybe help some readers benefit from the same. A lot of the content in this article is derived from learnopengl.com. In this article I'll cover all the steps involved in rendering a triangle with OpenGl excluding the installation steps. For refernece I am using Windows + Visual Studio as my setup. [ Read more ]

Fast and maintainable image processing in Android with Halide - Part 3

Halide is an open-source domain specific language designed to make it easier to write and maintain high-performance image processing or array processing code on modern machines. I have been writing a series on Halide and this article is 3rd one in the series. In this article I will be wrting about how to use Halide with Android. To assert on the performance benefits I am going to use the problem statement of YUV to RGB color format conversion. I have written couple of articles in the past showing different ways to do image processing in Android with this example. [ Read more ]

Errors in Qualcomm documentation on Halide for HVX

Qualcomm has a user guide called Halide for HVX (Hexagon Vector eXtension). The documentation is a great guide to both halide and HVX. I learned many things from it. But, besides all the good stuff, I found a couple of mistakes in the documentation which in the beginning confused me. I had to validate some of those to ensure if I had a wrong understanding before. Writing this article for 1) folks similar to me who got confused and 2) Qualcomm folks to correc the documentation. [ Read more ]

Write fast and maintainable code with Halide - Part 2

Halide is an open-source programming language designed to make it easier to write high-performance image processing or array processing code on modern machines. In the last article of this series, I explained about what Halide is, what is the need and how Halide addresses it. In this article I'll dig deeper and share some general concepts in Halide. This article is pretty much a fork of the documentation and tutorials shared by the Halide development team. I have tried to summarize some of the primary concepts for the readers and my own future reference. [ Read more ]

Write fast and maintainable code with Halide - Part 1

Halide is an open-source programming language designed to make it easier to write high-performance image processing or array processing code on modern machines. Rather than being a standalone programming language, Halide is embedded in C++. It currently targets different CPUs, Operating Systems, different compute APIs like CUDA, OpenGl, OpenCL etc.In this article, you can find more information on what Halide is, how to write image processing algorithms with Halide, how it is different from the rest and what kind of performance we can achieve with Halide. [ Read more ]

Processing images fast with native code in Android

If you are writing applications that processes large images captured with a camera or an existing image on the device you need to be extra careful. These days camera on phones are easily equipped with high resolution sensors. Let's look at a 13MP image. It has 13,000,000 pixels. Any operation that interact with every pixel needs like 13 million iterations. In my experience, its both easier and better to handle these complex image processing operations with native code very particularly to keep it performant. This is a very basic article demonstrating how to do image processing with native code in Android. I'll also show by an example that the performance of a very simple and unoptimized C++ code comes very close to fairly optimized Java code for the same problem statement. [ Read more ]

Some core properties of a camera sensor - what makes a good camera sensor?

Most of us today have a smartphone we can't keep our hands off of. And one of the most important features of the smartphone, is the camera. And many photography enthusiasts may have additional equipments like a DSLR or a mirrorless camera or even a GoPro or a drone. So how do we decide if a camera is good or bad? It depends on several components - like the main 'sensor' in the camera, the software stack (thats usually associated with manufacturing company), the processor which dictates the performance of the camera and so on. In this article I'll be explaining some of the less known properties of a camera sensor that could be used to evaluate the sensors used in the camera. [ Read more ]

QR and barcode scanner in React

mebjas/html5-qrcode is a fairly used open source library for implementing QR Code or barcode scanner in a web application. There are several developers who have been using it under webview for android projects as well. In this article I'll be explaining how to use html5-qrcode with React so it's easier for developers using popular React framework to these functionalities with ease. [ Read more ]

Lowpass, Highpass, Bandreject and Bandpass filters in image processing

Spatial domain and frequency domain filters are commonly classified into four types of filters — low-pass, high-pass, band-reject and band-pass filters. In this article I have notes, code examples and image output for each one of them. A lot of this is derived from the book Digital Image Processing — by Rafael C. Gonzalez & Richard E. Woods and can be used as quick refresher. I'll only be listing some key pointers, sample code and the result of running them. [ Read more ]

Intensity transformation and spatial filtering

I have been reading the book Digital Image Processing — by Rafael C. Gonzalez & Richard E. Woods and writing some notes to remember. But I realized I have done this before, but I don't know where the notebooks are. So here I am writing my notes in an article, chapter by chapter. This one is from chapter — 2 — Intensity transformation and spatial filtering. I'll only be listing some key pointers, sample code and the result of running them. For other readers this article may be interesting if you are looking for code examples on how to run a certain algorithm you read about in the chapter. [ Read more ]

How to get the latest country wise gold price in Google sheets

I use Google Sheets for tracking my expenses, assets, and liabilities. Google Sheet has a first-class support for querying the latest values of stocks on US-based exchanges like NASDAQ or NYSE. On the other hand it doesn't have support for getting spot price of metals and commodities which can vary market by market. In this article I have shared a hack on how to scrape some reliable sources of gold prices and get the latest values in Google sheets to easily track the latest value of gold by market. [ Read more ]

Guide C++ compiler to auto vectorise the code

C++ compilers these days have code optimization techniques like loop vectorizer which allows the compilers to generate vector instructions for code written in scalar format. It also depends on how the code is written for the compiler to understand if it can auto-vectorize the code or not. In this article I'll be sharing some ways for developers to guide the compiler to auto-vectorize a certain for loop in C++. It may not necessarily work for all compilers. The examples here are borrowed from ARM's documentation and applies to following compilers that can generate Neon code like GCC, LLVM-Clang, Arm C/C++ compiler and more. I got nearly 18% speedup with simple tricks. [ Read more ]

Faster image processing in Android Java using multi threading

While I was exploring different ways to do efficient image processing in Android I realized that a simple two-dimensional for-loop when written in Java vs C++ could have very different performance. For example: I have been comparing different ways we can do color-space conversion of an image in Android from YUV color-space to RGB and observed the Java version to be 4.62x slower than a standard C++ native code integrated with JNI. In this article, I am going to write about why do we see this kind of performance difference and how we could mitigate it. [ Read more ]

How to use RenderScript to convert YUV_420_888 YUV Image to Bitmap

RenderScript turns out to be one of the best APIs for running computationally-intensive code on the CPU or GPU (that too, without having to make use of the NDK or GPU-specific APIs). We can use some existing intrinsics or create our new kernels that describe the computation and the framework takes care of scheduling & execution. In this code I have explained how to useScriptIntrinsicYuvToRGB intrinsic that is available in Android APIs to convert an android.media.Image in YUV_420_888 format to Bitmap. [ Read more ]

Could not build wheels for argon2-cffi which use PEP 517 and cannot be installed directly - Jupyter Notebook Mac OS

While trying to install jupyter notebook in Mac OS with PIP you could get error like 'ERROR: Could not build wheels for argon2-cffi which use PEP 517 and cannot be installed directly'. This took me a while to fix and here's a quick writeup on how to address it based on my experience. [ Read more ]

Using BarcodeDecoder in javascript

In the past the APIs like MediaDevices.getUserMedia(), local file reading API made it fairly easy to capture images and process them. This made libraries like @mebjas/html5-qrcode possible - which today allows developers to easily integrate QR code or bar code scanning capabilities to their web applications. Several developers have used the library to build Android applications using Cordova. Now the Web Platform Incubator Community Group has come with a draft for shape detector APIs which aims at providing access to accelearated shape detectors (like human faces, QR codes, bar codes, etc) for still images and live image feeds. This is particularly exciting for mobile devices which today comes with hardware chips which are highly performant at certain tasks like detecting faces, barcodes, texts etc. In this article, I have explained how to use the new BarcodeDetector library and how it can be used directly with html5-qrcode and what kind of performance improvement we see when compared to ZXing js library. [ Read more ]

QR and barcode scanner using HTML and Javascript

The little QR code scanning library I have been maintaining since 2015 has been getting more attention recently. And with power came responsibilities, bugs, and feature requests. Some of the key features requested by developers were more reliable scanning and the ability to scan different types of bar codes. With version 2.0.0 onwards developers can scan different types of 1D codes (bar codes) and 2D codes (like QR codes or AZTEC).

This article lists out everything new in version 2.x.x. I'll also list out the new APIs and capabilities that developers can use to integrate a more powerful code scanning capability to their web pages or apps. [ Read more ]

Joyalukkas gold scheme Singapore - is it a good investment? an analysis.

Joyalukkas is a fairly well-known certified jewelry group based out of India. They have a variety of gold schemes that allow customers to pre-plan for the future buy of gold and in a way invest in it. In Singapore, the scheme is as follows:

1. Invest a fixed amount say 1000 SGD every month for 12 month.
2. In the end of 12th month you can buy gold worth 12000 SGD + 1000 SGD.

This was introduced to me by a friend and in the beginning, this sounded like a no-brainer. This looked like a 8.3% ROI (with naive math) - which is greater than most of the instruments available to us for investment. I went down to the stores to understand more and finally decided it to not be a good investment vehicle. In this article, In this article, I intend to share my findings on why it turned out to be not as attractive as it looks. [ Read more ]

How to get the latest commodity pricing in Google Sheet

I use Google Sheets for tracking my expenses, assets, and liabilities. Google Sheet has a first-class support for querying the latest values of stocks on US-based exchanges like NASDAQ or NYSE. On the other hand, it doesn't have as well support for other exchanges like SGX: Singapore Exchange. To overcome this we are supposed to run awkward hacks like crawling yahoo finance pages. In this article I'll be writing about how to write an AppScript that will allow you to crawl such information from some API repository and mildly mention the option to cache some data if there is a limited API call per hour. I'll be using the example of how to get the latest gold price, silver price, and platinum price in USD in Google Sheet. This can be easily extended to solve questions on how to get the latest gold prices in India in Google Sheet or the latest price of Indian stocks or Singapore Exchange stocks. [ Read more ]

How to use YUV (YUV_420_888) Image in Android

ImageFormat#YUV_420_888 is one of the most common image format supported by Android Cameras. It's a multi-plane YUV (YCbCr) format represented by three separate planes in android.media.Image. This format can be used for processing the input frames before saving to disk or some other action. A very common question around YUV is how to consume it in Android. In this article, I'd describe different ways it can be used. The most common question is how to convert YUV to Bitmap or jpeg format in Android? [ Read more ]

Html5QrcodeScanner - End to end QR Code scanner for web, not just a library

When I started the project html5-qrcode, my goal was to make it easier to implement QR code scanning in web applications. I wanted to abstract the inner details of how the camera is accessed in HTML5 and how it's connected with a scanning library. Some developers started to adopt the library and use it in their products. In general, the adoption trend seemed to be users trying to replicate the demo code. The library was stateful and required a series of steps to connect it with UI. In the latest version, I have implemented another wrapper called Html5QrcodeScanner which enable developers to integrate QR Code scanning with ~5 lines of code. No more statefulness! [ Read more ]

Exploring Android 11 developer preview - Part 3

Android released the Developer Preview 3 for Android 11 on 23rd April 2020 with a blog post from Dave Burke, VP of Engineering. The release is for developers only and not ready for daily or consumer use-cases. If you haven't read about the features made available in Developer Preview 1 or Developer preview 2, I recommend reading them first. In this article, I'll be exploring the updates in developer preview 3 and user-visible changes I could spot by playing with it on the device. To make flashing a bit easier, they have added the support for flashing the preview release from the Android Flash Tool. [ Read more ]

HTML5 QR Code scanning with javascript - Support for scanning the local file and using default camera added (v1.0.5)

html5-qrcode helps developer to integrate QR code scanning in HTML5 based applications. It abstracts camera access and usage on different browsers and simplify QR Code scanning for developers. The primary goal of the library is cross-platform support across different OS versions and Browsers. One of the key problems with cross-platform support was some browsers in Android and many browsers in iOS (almost all other than Safari which are based on WebKit for iOS) did not support camera access. This prevents users from doing inline QR Code scanning which is the primary feature of the library. To mitigate this I have added support for scanning local media on the device and it implicitly adds support for capturing QR Code using default camera on the device and scanning it. This is an upgrade to the existing library - you can read more about it here. In this article I have explained now file-based scanning works and how to use it. [ Read more ]

HTML5 QR Code scanning with javascript - launched v1.0.1

In 2015 I had written an HTML5 based QR code scanning library as a jQuery extension. Recently I realised there was some consistent traffic on my Github Project and the demo page. As I dug more into what was going on and I was embarrassed to see the poor design and obsolete support to the latest HTML APIs around Camera. I recently fixed some of the issues and refactored the javascript library that is now independent of jQuery and supports Promise based APIs. In this article I'll explain how to use the new version of the library, some changes and reasons for them and existing issues and plan to fix them. [ Read more ]

Exploring Android 11 developer preview - Part 2

Android released the Developer Preview 2 for Android 11 on 18th March 2020 with a blog post from Dave Burke, VP of Engineering. The release is for developers only and not ready for daily or consumer use-cases. If you haven't read about the features made available in Developer Preview 1, I recommend reading that first - Exploring Android 11 developer preview - Part 1. In this article, I'll be exploring the updates in DP2 and user-visible changes I could spot by playing with the device. To make flashing a bit easier, they have added the support for flashing the preview release from the Android Flash Tool. [ Read more ]

Memory sharing in Linux with MMAP

MMAP is a UNIX system call that maps files into memory. It's a method used for memory-mapped file I/O. It brings in the optimization of lazy loading or demand paging such that the I/O or reading file doesn't happen when the memory allocation is done, but when the memory is accessed.
In this article, I'll be explaining how what mmap is and how it can be used for sharing memory in Linux. It kind of is the backbone of shared memory in Android. [ Read more ]

Exploring Android 11 developer preview - Part 1

Android has released a developer preview of Android 11 (aka Android R) for the Pixel series (and emulator) last week. As per the blog article, many changes have been introduced to support upcoming technologies like 5G, foldable displays, machine learning APIs and to strengthen Android's position in privacy and security. On top of that, I could see a couple of additions to support ongoing trends like pinhole or waterfall screens or giving first-class support for bokeh in Camera. In this article, I'd be exploring some of these changes and explaining what it means for users and developers. [ Read more ]

Android Camera Hardware Architecture explained

Android is the most popular mobile operating system in the market today. Towards the end of the last decade, the camera has emerged as one of the most important factors that contribute towards smartphone sales and different OEMs are trying to stay at the top of the throne. Smartphone cameras are built very similar to digital cameras in a compact package. With high-end processing capabilities it has taken a big leap and is competing with digital cameras today. In this post, I'll be covering the general hardware architecture of modern Android Camera. [ Read more ]

Android Camera Subsystem - basic image processing steps done at hardware level in Android Camera

Towards the end of last decade camera has emerged as one of the key factors that contribute towards smartphone sales and different manufacturers are trying to stay at the top of the throne. While everyone loves using a camera - the set of things that happens in a different layer of the hardware and software stack of Android Camera is not very popular knowledge. The design, implementation, and usage of these systems govern metrics like image quality and camera performance (latency, throughput, etc). In this post, I'll be describing various algorithms implemented in the android camera subsystem that is consistent across OEMs which are used to produce stunning images that we capture from camera applications. [ Read more ]

QR Code scanner using HTML5 and Javascript

LazarSoft had written an excellent set of QR code decoding javascript libraries. However, it still requires some efforts to integrate the work with a web project, as we need to take care of enumerating different camera devices, setting up canvas or the video element and hooking the stream with qrcode library. Also recently, in leu of privacy concers, some javascript APIs related to accessing the camera has changed and it requires some efforts to ensure different platforms like mobile or PC, chrome or safari support it. To reduce these pain points I wrote a jQuery based wrapper on to abstract tasks like camera enumeration and UI setup - it can be found at mebjas/html5-qrcode. In this article I have documented some details on how to use it and what is happening under the hook. [ Read more ]

Interview tips 1 – In a coding interview use abstractions!

While I was working with Microsoft, I had opportunity to take around 10 – 20 coding or design interviews (candidates with 1 – 12 years of experience). There was this one thing that I felt was too common, which would lead to bad impression and potential rejection – it’s that the candidate attempts to solve every part of the question rather than focusing on what is important. This is particularly prevalent amongst candidates who chose C / C++ as programming language of choice. And, this was common irrespective of years of experience.
In this article I have shared the problem and my opinion on how to tackle it. [ Read more ]

Bypass cell size limitation (64KB) of Azure Storage Table

Azure Storage Table is Azure’s NoSQL offering to store large number of data (upto peta bytes) in a table (tables have rows and columns). It provides indexing on two keys namely partition key and row key which allows you to retrieve data efficiently. No other columns are indexed, but that’s expected behavior. It has limitations on size of a single row (1 MB), size of a single column (64KB), number of columns per row (255) and so on. While I don’t really know why some of them exist – they just do. In this article I have proposed a naughty but elegant way of bypassing this issue. [ Read more ]

Efficiently writing large objects to Azure Blob Storage – JSON, Compression, Caching, Streaming in .Net

Azure Storage Blob is an Azure Storage offering that allows you to store giga bytes of data in from hundreds to billions of objects in hot, cool, or archive tiers, depending on how often data access is needed. Store any type of unstructured data—images, videos, audio, documents and more—easily and cost-effectively. These features make it a strong candidate for storing serialized Machine Learning Models if you have models per tenant. But, if the application is READ + WRITE heavy and the size of objects are large it can lead to a variety of issues, primarily:

1. Large I/O calls
2. OutOfMemory Exceptions (OOM)

In this article I have explained why the issues occur and what are the ways to mitigate them. [ Read more ]

Support for custom logging in CSRF Protector Library and more

Added some new feature to CSRF Protector library (version 1.0.1) like:

1. Support for custom logger.
2. X-CSRF-Protection removed from response header.
3. Options added to make CSRF Token in cookie https only and it’s expiry time configurable.
4. More...

[ Read more ]

Compress sequence of UNIX timestamps with microseconds accuracy to ~10bits/timestamp

I was given a problem statement by interviewer: There is a stream of timestamps that need to be transferred across some network stream. The goal is to compress this sequence of timestamps with microseconds accuracy, in lossless fashion. Also the encoding and decoding process should be very fast so that it can scale for time critical processes..
I found the problem statement very intriguing and went through with it and it was a an amazing learning experience. In this article I have explained the problem statemenet and how I solved it. [ Read more ]

Introducing minor improvements to CSRF Protector PHP

The OWASP CSRF Protector project started with an aim to develop a solution that can mitigate Cross Site Request Forgery in web applications without much developer effort. We have recently released v1.0.0 of CSRF Protector PHP. These improvements / fixes were based on issues raised by different users. Here’s the two major changes that we have introduced and why:

1. Added support for application/json content-type.
2. Made path, domain and secure property of CSRF Token Cookie configurable.

[ Read more ]

A fault tolerant distributed key value store from scratch

We had a B-Tech course on Distributed Systems and I took a course on on Cloud Computing Concepts 1 by Dr Indranil Gupta (UIUC) a year back and for long, I have been thinking about trying out different concepts explained in the course together as something meaningful. In this article I have attempted to describe how to build a fault tolerant distributed key value store from scratch. A key-value store, or key-value database, is a data storage paradigm designed for storing, retrieving, and managing associative arrays, a data structure more commonly known today as a dictionary or hash. A distributed Key Value store is one where data is replicated across different nodes such that there is High Availability and No single point of failure [ Read more ]

Logging out and then logging in throws 403 error with CSRF Protector PHP – fix / workaround

Recently an interesting bug came up in CSRF Protector PHP. If you log out of your website and then try to login again there only, CSRF Protector throws 403 – forbidden response. So this comes by design because first thing that you do in your logout script is, initiate CSRF Protector > let it do it’s stuff and then destroy session to logout the user. Now this screws everything because CSRFP is dependent on tokens it store in session variables. So next time you try to login again which is a POST request, it’s unable to validate the incoming token and throws 403 or whatever is the failedValdiationResponse in your config. [ Read more ]

CSRF protector – Concept, Design and Future

CSRF has been on OWASP Top 10 for a long time, and several methods have been implemented to mitigate it. Most of the web frameworks have inbuilt methods, which can be used to make a website safe against CSRF. However lot of web applications are still vulnerable to CSRF and its partially because developer tend to forget to implement it with every FORM or AJAX requests. And that is because the approach used for CSRF Mitigation in most of the frameworks is not centralised, we have to attach a token with every FORM we want to protect. OWASP CSRF Protector on the other hand used centralised approach, it employs javascript on client side to ensure, tokens are sent with every request sent from client, so that is correctly validated on server. This saves developer from adding a token everywhere. I’ll discuss more about the concept later in this article. [ Read more ]

Drawing a 3d surface patch using Bezier equations

This post covers:

• What’s a bezier equation?
• Creating a 3d surface patch using Bezier curves
• Making it interactive

Its a cubic parametric curve used to draw smooth curves, we can draw a bezier curve using two endpoints and two control points. [ Read more ]