Dec 12, 2023
•
android
java
multitheading
image-processing
optimisation
Different ways to do image processing in Android
[ Read more ]
Aug 1, 2023
•
AI
document-AI
transformers
BERT
Document AI refers to techniques for automatically reading, understanding, and analyzing business documents or visual documents in general. Visual documents are electronic documents like a reciept or pdf document or photograph of an ID card. In this article I will be sharing my notes on the paper - 'LayoutLM: Pre-training of Text and Layout for Document Image Understanding' by Yiheng Xu et. al. When the paper was published the authors claimed this to be SOTA on multiple Visual Document Understanding tasks.
[ Read more ]
Dec 9, 2022
•
android
java
Image
ImageWriter
Camera2
JPEG
YUV_420_888
Android framework provides ImageWriter class as a way for applications to produce Image data int a Surface which can then be consumed by other components like CameraDevice. I found a couple of hiccups while trying to write jpeg images to ImageWriter buffers and found that there is scarce documentation around this out in the wild. In this article I'll share briefly on how to write jpeg images or YUV images to ImageWriter in Android.
[ Read more ]
Nov 12, 2022
•
android
c++
ndk
file
image
image-processing
Software development in Android can be done using the Java SDK or Native Development Kit aka NDK made available by Android Open Source Project (AOSP). NDK is often used for writing high performance code like image processing algorithms. There may be times when you would want to load image from disk.
In this article I'll cover how to read an image file in C++ with NDK and JNI. The approach can in general be used to load any file in native layer.
[ Read more ]
May 22, 2022
•
graphics
opengl
c++
Learning OpenGL has been a long time todo and I have gladly taken a step 1. But I realised it's a lot of steps to just to get to hello world application or in this case I must say - 'Hello Traiangle' application. In the past writing a blog article has helped me get a better grasp of things, have a easy to access notes and maybe help some readers benefit from the same. A lot of the content in this article is derived from learnopengl.com. In this article I'll cover all the steps involved in rendering a triangle with OpenGl excluding the installation steps. For refernece I am using Windows + Visual Studio as my setup.
[ Read more ]
May 8, 2022
•
optimisation
halide
c++
performance
maintenance
android
jni
Halide is an open-source domain specific language designed to make it easier to write and maintain high-performance image processing or array processing code on modern machines. I have been writing a series on Halide and this article is 3rd one in the series. In this article I will be wrting about how to use Halide with Android. To assert on the performance benefits I am going to use the problem statement of YUV to RGB color format conversion. I have written couple of articles in the past showing different ways to do image processing in Android with this example.
[ Read more ]
May 3, 2022
•
optimisation
halide
c++
performance
maintenance
android
jni
Qualcomm has a user guide called Halide for HVX (Hexagon Vector eXtension). The documentation is a great guide to both halide and HVX. I learned many things from it. But, besides all the good stuff, I found a couple of mistakes in the documentation which in the beginning confused me. I had to validate some of those to ensure if I had a wrong understanding before. Writing this article for 1) folks similar to me who got confused and 2) Qualcomm folks to correc the documentation.
[ Read more ]
Apr 10, 2022
•
optimisation
halide
c++
performance
maintainance
Halide is an open-source programming language designed to make it easier to write high-performance image processing or array processing code on modern machines. In the last article of this series, I explained about what Halide is, what is the need and how Halide addresses it. In this article I'll dig deeper and share some general concepts in Halide. This article is pretty much a fork of the documentation and tutorials shared by the Halide development team. I have tried to summarize some of the primary concepts for the readers and my own future reference.
[ Read more ]
Apr 9, 2022
•
optimisation
halide
c++
performance
maintainance
Halide is an open-source programming language designed to make it easier to write high-performance image processing or array processing code on modern machines. Rather than being a standalone programming language, Halide is embedded in C++. It currently targets different CPUs, Operating Systems, different compute APIs like CUDA, OpenGl, OpenCL etc.In this article, you can find more information on what Halide is, how to write image processing algorithms with Halide, how it is different from the rest and what kind of performance we can achieve with Halide.
[ Read more ]
Jan 1, 2022
•
android
camera2
YUV_420_888
native
YUV
Bitmap
optimisation
JNI
c++
If you are writing applications that processes large images captured with a camera or an existing image on the device you need to be extra careful. These days camera on phones are easily equipped with high resolution sensors. Let's look at a 13MP image. It has 13,000,000 pixels. Any operation that interact with every pixel needs like 13 million iterations. In my experience, its both easier and better to handle these complex image processing operations with native code very particularly to keep it performant. This is a very basic article demonstrating how to do image processing with native code in Android. I'll also show by an example that the performance of a very simple and unoptimized C++ code comes very close to fairly optimized Java code for the same problem statement.
[ Read more ]
Dec 18, 2021
•
camera
sensor
computational-photography
optics
Most of us today have a smartphone we can't keep our hands off of. And one of the most important features of the smartphone, is the camera. And many photography enthusiasts may have additional equipments like a DSLR or a mirrorless camera or even a GoPro or a drone. So how do we decide if a camera is good or bad? It depends on several components - like the main 'sensor' in the camera, the software stack (thats usually associated with manufacturing company), the processor which dictates the performance of the camera and so on. In this article I'll be explaining some of the less known properties of a camera sensor that could be used to evaluate the sensors used in the camera.
[ Read more ]
Nov 21, 2021
•
html
javascript
react
qrcode
barcode
qrcode-scanner
barcode-scanner
mebjas/html5-qrcode is a fairly used open source library for implementing QR Code or barcode scanner in a web application. There are several developers who have been using it under webview for android projects as well. In this article I'll be explaining how to use html5-qrcode with React so it's easier for developers using popular React framework to these functionalities with ease.
[ Read more ]
Sep 12, 2021
•
image-processing
spatial-filters
Spatial domain and frequency domain filters are commonly classified into four types of filters — low-pass, high-pass, band-reject and band-pass filters. In this article I have notes, code examples and image output for each one of them. A lot of this is derived from the book Digital Image Processing — by Rafael C. Gonzalez & Richard E. Woods and can be used as quick refresher. I'll only be listing some key pointers, sample code and the result of running them.
[ Read more ]
Sep 11, 2021
•
image-processing
spatial-filters
I have been reading the book Digital Image Processing — by Rafael C. Gonzalez & Richard E. Woods and writing some notes to remember. But I realized I have done this before, but I don't know where the notebooks are. So here I am writing my notes in an article, chapter by chapter. This one is from chapter — 2 — Intensity transformation and spatial filtering. I'll only be listing some key pointers, sample code and the result of running them. For other readers this article may be interesting if you are looking for code examples on how to run a certain algorithm you read about in the chapter.
[ Read more ]
Sep 1, 2021
•
finance
app-script
google-sheet
commodity
gold-price
I use Google Sheets for tracking my expenses, assets, and liabilities. Google Sheet has a first-class support for querying the latest values of stocks on US-based exchanges like NASDAQ
or NYSE
. On the other hand it doesn't have support for getting spot price of metals and commodities which can vary market by market. In this article I have shared a hack on how to scrape some reliable sources of gold prices and get the latest values in Google sheets to easily track the latest value of gold by market.
[ Read more ]
Aug 21, 2021
•
android
C++
neon
auto-vectorization
C++ compilers these days have code optimization techniques like loop vectorizer
which allows the compilers to generate vector instructions for code written in scalar format. It also depends on how the code is written for the compiler to understand if it can auto-vectorize the code or not. In this article I'll be sharing some ways for developers to guide the compiler to auto-vectorize a certain for loop in C++. It may not necessarily work for all compilers. The examples here are borrowed from ARM's documentation and applies to following compilers that can generate Neon code like GCC, LLVM-Clang, Arm C/C++ compiler and more. I got nearly 18% speedup with simple tricks.
[ Read more ]
Jul 24, 2021
•
android
java
multitheading
image-processing
optimisation
While I was exploring different ways to do efficient image processing in Android I realized that a simple two-dimensional for-loop when written in Java vs C++ could have very different performance. For example: I have been comparing different ways we can do color-space conversion of an image in Android from YUV color-space to RGB and observed the Java version to be 4.62x slower than a standard C++ native code integrated with JNI. In this article, I am going to write about why do we see this kind of performance difference and how we could mitigate it.
[ Read more ]
Jul 11, 2021
•
android
camera2
YUV_420_888
android.media.Image
YUV
Bitmap
optimisation
RenderScript
ScriptIntrinsicYuvToRGB
RenderScript turns out to be one of the best APIs for running computationally-intensive code on the CPU or GPU (that too, without having to make use of the NDK or GPU-specific APIs). We can use some existing intrinsics or create our new kernels that describe the computation and the framework takes care of scheduling & execution. In this code I have explained how to useScriptIntrinsicYuvToRGB
intrinsic that is available in Android APIs to convert an android.media.Image
in YUV_420_888
format to Bitmap
.
[ Read more ]
Jul 6, 2021
•
Mac
MacOS
jupyter
notebook
argon2-cffi
troubleshooting
While trying to install jupyter notebook
in Mac OS with PIP you could get error like 'ERROR: Could not build wheels for argon2-cffi which use PEP 517 and cannot be installed directly'. This took me a while to fix and here's a quick writeup on how to address it based on my experience.
[ Read more ]
Jun 20, 2021
•
html
javascript
qr-code
camera
barcode
In the past the APIs like MediaDevices.getUserMedia()
, local file reading API made it fairly easy to capture images and process them. This made libraries like @mebjas/html5-qrcode possible - which today allows developers to easily integrate QR code or bar code scanning capabilities to their web applications. Several developers have used the library to build Android applications using Cordova. Now the Web Platform Incubator Community Group has come with a draft for shape detector APIs
which aims at providing access to accelearated shape detectors (like human faces, QR codes, bar codes, etc) for still images and live image feeds. This is particularly exciting for mobile devices which today comes with hardware chips which are highly performant at certain tasks like detecting faces, barcodes, texts etc. In this article, I have explained how to use the new BarcodeDetector
library and how it can be used directly with html5-qrcode and what kind of performance improvement we see when compared to ZXing js library
.
[ Read more ]
Jun 14, 2021
•
html
javascript
typescript
qr-code
camera
barcode
The little QR code scanning library I have been maintaining since 2015 has been getting more attention recently. And with power came responsibilities, bugs, and feature requests. Some of the key features requested by developers were more reliable scanning and the ability to scan different types of bar codes. With version 2.0.0
onwards developers can scan different types of 1D codes (bar codes) and 2D codes (like QR codes or AZTEC).
This article lists out everything new in version 2.x.x
. I'll also list out the new APIs and capabilities that developers can use to integrate a more powerful code scanning capability to their web pages or apps.
[ Read more ]
Jan 24, 2021
•
finance
joyalukkas
analysis
gold-scheme
gold
Joyalukkas is a fairly well-known certified jewelry group based out of India. They have a variety of gold schemes that allow customers to pre-plan for the future buy of gold and in a way invest in it. In Singapore, the scheme is as follows:
- Invest a fixed amount say 1000 SGD every month for 12 month.
- In the end of 12th month you can buy gold worth 12000 SGD + 1000 SGD.
This was introduced to me by a friend and in the beginning, this sounded like a no-brainer. This looked like a 8.3% ROI
(with naive math) - which is greater than most of the instruments available to us for investment. I went down to the stores to understand more and finally decided it to not be a good investment vehicle. In this article, In this article, I intend to share my findings on why it turned out to be not as attractive as it looks.
[ Read more ]
Dec 30, 2020
•
finance
app-script
google-sheet
commodity
gold-price
silver-price
I use Google Sheets for tracking my expenses, assets, and liabilities. Google Sheet has a first-class support for querying the latest values of stocks on US-based exchanges like NASDAQ
or NYSE
. On the other hand, it doesn't have as well support for other exchanges like SGX: Singapore Exchange
. To overcome this we are supposed to run awkward hacks like crawling yahoo finance
pages. In this article I'll be writing about how to write an AppScript
that will allow you to crawl such information from some API repository and mildly mention the option to cache some data if there is a limited API call per hour. I'll be using the example of how to get the latest gold price, silver price, and platinum price in USD in Google Sheet. This can be easily extended to solve questions on how to get the latest gold prices in India in Google Sheet or the latest price of Indian stocks or Singapore Exchange stocks.
[ Read more ]
Aug 27, 2020
•
android
camera
camera2
YUV_420_888
android.media.Image
YUV
JPEG
Bitmap
optimisation
c++
ndk
ImageFormat#YUV_420_888 is one of the most common image format supported by Android Cameras. It's a multi-plane YUV (YCbCr) format represented by three separate planes in android.media.Image. This format can be used for processing the input frames before saving to disk or some other action. A very common question around YUV is how to consume it in Android. In this article, I'd describe different ways it can be used. The most common question is how to convert YUV to Bitmap or jpeg format in Android?
[ Read more ]
Jun 27, 2020
•
html
javascript
jquery
qrcode
camera
e2e
When I started the project html5-qrcode, my goal was to make it easier to implement QR code scanning in web applications. I wanted to abstract the inner details of how the camera is accessed in HTML5 and how it's connected with a scanning library. Some developers started to adopt the library and use it in their products. In general, the adoption trend seemed to be users trying to replicate the demo code. The library was stateful and required a series of steps to connect it with UI. In the latest version, I have implemented another wrapper called Html5QrcodeScanner which enable developers to integrate QR Code scanning with ~5 lines of code. No more statefulness!
[ Read more ]
May 3, 2020
•
android
android-11
android-R
privacy
security
camera
Android released the Developer Preview 3 for Android 11 on 23rd April 2020 with a blog post from Dave Burke, VP of Engineering. The release is for developers only and not ready for daily or consumer use-cases. If you haven't read about the features made available in Developer Preview 1 or Developer preview 2, I recommend reading them first. In this article, I'll be exploring the updates in developer preview 3 and user-visible changes I could spot by playing with it on the device. To make flashing a bit easier, they have added the support for flashing the preview release from the Android Flash Tool.
[ Read more ]
Apr 18, 2020
•
html
javascript
jquery
qrcode
camera
promise
html5-qrcode helps developer to integrate QR code scanning in HTML5 based applications. It abstracts camera access and usage on different browsers and simplify QR Code scanning for developers. The primary goal of the library is cross-platform support across different OS versions and Browsers. One of the key problems with cross-platform support was some browsers in Android and many browsers in iOS (almost all other than Safari which are based on WebKit for iOS) did not support camera access. This prevents users from doing inline QR Code scanning which is the primary feature of the library. To mitigate this I have added support for scanning local media on the device and it implicitly adds support for capturing QR Code using default camera on the device and scanning it. This is an upgrade to the existing library - you can read more about it here. In this article I have explained now file-based scanning works and how to use it.
[ Read more ]
Apr 4, 2020
•
html
javascript
jquery
qrcode
camera
promise
In 2015 I had written an HTML5 based QR code scanning library as a jQuery extension. Recently I realised there was some consistent traffic on my Github Project and the demo page. As I dug more into what was going on and I was embarrassed to see the poor design and obsolete support to the latest HTML APIs around Camera. I recently fixed some of the issues and refactored the javascript library that is now independent of jQuery and supports Promise based APIs. In this article I'll explain how to use the new version of the library, some changes and reasons for them and existing issues and plan to fix them.
[ Read more ]
Mar 20, 2020
•
android
android-11
android-R
privacy
security
camera
Android released the Developer Preview 2 for Android 11 on 18th March 2020 with a blog post from Dave Burke, VP of Engineering. The release is for developers only and not ready for daily or consumer use-cases. If you haven't read about the features made available in Developer Preview 1, I recommend reading that first - Exploring Android 11 developer preview - Part 1. In this article, I'll be exploring the updates in DP2 and user-visible changes I could spot by playing with the device. To make flashing a bit easier, they have added the support for flashing the preview release from the Android Flash Tool.
[ Read more ]
Mar 15, 2020
•
android
linux
memory
shared memory
mmap
munmap
malloc
MMAP is a UNIX system call that maps files into memory. It's a method used for memory-mapped file I/O. It brings in the optimization of lazy loading or demand paging such that the I/O or reading file doesn't happen when the memory allocation is done, but when the memory is accessed.
In this article, I'll be explaining how what mmap is and how it can be used for sharing memory in Linux. It kind of is the backbone of shared memory in Android.
[ Read more ]
Feb 22, 2020
•
android
android-11
android-R
privacy
security
camera
Android has released a developer preview of Android 11 (aka Android R) for the Pixel series (and emulator) last week. As per the blog article, many changes have been introduced to support upcoming technologies like 5G, foldable displays, machine learning APIs and to strengthen Android's position in privacy and security. On top of that, I could see a couple of additions to support ongoing trends like pinhole or waterfall screens or giving first-class support for bokeh in Camera. In this article, I'd be exploring some of these changes and explaining what it means for users and developers.
[ Read more ]
Jan 31, 2020
•
android
android-camera
hal
image-processing
hardware
isp
dsp
lens
sensor
Android is the most popular mobile operating system in the market today. Towards the end of the last decade, the camera has emerged as one of the most important factors that contribute towards smartphone sales and different OEMs are trying to stay at the top of the throne. Smartphone cameras are built very similar to digital cameras in a compact package. With high-end processing capabilities it has taken a big leap and is competing with digital cameras today. In this post, I'll be covering the general hardware architecture of modern Android Camera.
[ Read more ]
Jan 22, 2020
•
android
android-camera
hal
image-processing
computational-photograhpy
Towards the end of last decade camera has emerged as one of the key factors that contribute towards smartphone sales and different manufacturers are trying to stay at the top of the throne. While everyone loves using a camera - the set of things that happens in a different layer of the hardware and software stack of Android Camera is not very popular knowledge. The design, implementation, and usage of these systems govern metrics like image quality and camera performance (latency, throughput, etc). In this post, I'll be describing various algorithms implemented in the android camera subsystem that is consistent across OEMs which are used to produce stunning images that we capture from camera applications.
[ Read more ]
Dec 30, 2019
•
html
javascript
jquery
qrcode
camera
LazarSoft had written an excellent set of QR code decoding javascript libraries. However, it still requires some efforts to integrate the work with a web project, as we need to take care of enumerating different camera devices, setting up canvas
or the video
element and hooking the stream with qrcode library
. Also recently, in leu of privacy concers, some javascript APIs related to accessing the camera has changed and it requires some efforts to ensure different platforms like mobile or PC, chrome or safari support it. To reduce these pain points I wrote a jQuery based wrapper on to abstract tasks like camera enumeration and UI setup - it can be found at mebjas/html5-qrcode. In this article I have documented some details on how to use it and what is happening under the hook.
[ Read more ]
May 7, 2019
•
dotnet
interview
microsoft
python
interview-tips
While I was working with Microsoft, I had opportunity to take around 10 – 20 coding or design interviews (candidates with 1 – 12 years of experience). There was this one thing that I felt was too common, which would lead to bad impression and potential rejection – it’s that the candidate attempts to solve every part of the question rather than focusing on what is important. This is particularly prevalent amongst candidates who chose C / C++ as programming language of choice. And, this was common irrespective of years of experience.
In this article I have shared the problem and my opinion on how to tackle it.
[ Read more ]
Jan 6, 2019
•
architecture
system-design
azure
azure-storage-table
compression
distributed-systems
microsoft
open-source
cloud
Azure Storage Table is Azure’s NoSQL offering to store large number of data (upto peta bytes) in a table (tables have rows and columns). It provides indexing on two keys namely partition key and row key which allows you to retrieve data efficiently. No other columns are indexed, but that’s expected behavior. It has limitations on size of a single row (1 MB), size of a single column (64KB), number of columns per row (255) and so on. While I don’t really know why some of them exist – they just do. In this article I have proposed a naughty but elegant way of bypassing this issue.
[ Read more ]
Apr 3, 2018
•
architecture
system-design
azure
azure-storage-blob
compression
distributed-systems
machine-learning
microsoft
Azure Storage Blob is an Azure Storage offering that allows you to store giga bytes of data in from hundreds to billions of objects in hot, cool, or archive tiers, depending on how often data access is needed. Store any type of unstructured data—images, videos, audio, documents and more—easily and cost-effectively. These features make it a strong candidate for storing serialized Machine Learning Models if you have models per tenant. But, if the application is READ + WRITE heavy and the size of objects are large it can lead to a variety of issues, primarily:- Large I/O calls
- OutOfMemory Exceptions (OOM)
In this article I have explained why the issues occur and what are the ways to mitigate them.
[ Read more ]
Mar 17, 2018
•
csrf
javascript
open-source
owasp
security
web-security
php
Added some new feature to CSRF Protector library (version 1.0.1) like:- Support for custom logger.
- X-CSRF-Protection removed from response header.
- Options added to make CSRF Token in cookie https only and it’s expiry time configurable.
- More...
[ Read more ]
Nov 10, 2017
•
compression
mathematics
encoding
decoding
bits-manipulation
I was given a problem statement by interviewer: There is a stream of timestamps that need to be transferred across some network stream. The goal is to compress this sequence of timestamps with microseconds accuracy, in lossless fashion. Also the encoding and decoding process should be very fast so that it can scale for time critical processes..
I found the problem statement very intriguing and went through with it and it was a an amazing learning experience. In this article I have explained the problem statemenet and how I solved it.
[ Read more ]
Oct 7, 2017
•
csrf
javascript
open-source
owasp
security
web-security
php
The OWASP CSRF Protector project started with an aim to develop a solution that can mitigate Cross Site Request Forgery in web applications without much developer effort. We have recently released v1.0.0 of CSRF Protector PHP. These improvements / fixes were based on issues raised by different users. Here’s the two major changes that we have introduced and why:- Added support for application/json content-type.
- Made path, domain and secure property of CSRF Token Cookie configurable.
[ Read more ]
Sep 5, 2017
•
distributed-systems
hackathon
key-value-store
nodejs
We had a B-Tech course on Distributed Systems and I took a course on on Cloud Computing Concepts 1 by Dr Indranil Gupta (UIUC) a year back and for long, I have been thinking about trying out different concepts explained in the course together as something meaningful. In this article I have attempted to describe how to build a fault tolerant distributed key value store from scratch. A key-value store, or key-value database, is a data storage paradigm designed for storing, retrieving, and managing associative arrays, a data structure more commonly known today as a dictionary or hash. A distributed Key Value store is one where data is replicated across different nodes such that there is High Availability and No single point of failure
[ Read more ]
Nov 4, 2016
•
csrf
javascript
open-source
owasp
security
web-security
php
Recently an interesting bug came up in CSRF Protector PHP. If you log out of your website and then try to login again there only, CSRF Protector throws 403 – forbidden response. So this comes by design because first thing that you do in your logout script is, initiate CSRF Protector > let it do it’s stuff and then destroy session to logout the user. Now this screws everything because CSRFP is dependent on tokens it store in session variables. So next time you try to login again which is a POST request, it’s unable to validate the incoming token and throws 403 or whatever is the failedValdiationResponse in your config.
[ Read more ]
Nov 28, 2015
•
csrf
javascript
open-source
owasp
security
web-security
php
apache
CSRF has been on OWASP Top 10 for a long time, and several methods have been implemented to mitigate it. Most of the web frameworks have inbuilt methods, which can be used to make a website safe against CSRF. However lot of web applications are still vulnerable to CSRF and its partially because developer tend to forget to implement it with every FORM or AJAX requests. And that is because the approach used for CSRF Mitigation in most of the frameworks is not centralised, we have to attach a token with every FORM we want to protect. OWASP CSRF Protector on the other hand used centralised approach, it employs javascript on client side to ensure, tokens are sent with every request sent from client, so that is correctly validated on server. This saves developer from adding a token everywhere. I’ll discuss more about the concept later in this article.
[ Read more ]
Nov 18, 2014
•
graphics
html-canvas
javascript
mathematics
This post covers:- What’s a bezier equation?
- Creating a 3d surface patch using Bezier curves
- Making it interactive
Its a cubic parametric curve used to draw smooth curves, we can draw a bezier curve using two endpoints and two control points.
[ Read more ]