tech talks at applozic

Understanding Word Embeddings – Tech Talks at Applozic

Understanding how different words are related based on the context they are used is an easy task for us humans. If we take an example of articles then we read a lot of articles related to different topics. In almost all the articles where an author is trying to teach you a new concept then the author will try to use the examples which are already known to you, to teach you any new concept. In a similar way computer also needs a way where they can learn about a topic and where they can understand how different words are related.

Let me begin with the concept of language, these amazing different languages we have. We use that to communicate with each other and share different ideas.

But how do we explain a language in a better way? Some time back I was reading this book “Sapiens”, where the author goes back and questions a lot of things and in that, his explanation of language was good:

We can connect a limited number of sounds and signs to produce an infinite number of sentences, each with a distinct meaning.
We can thereby ingest, store and communicate a prodigious amount of information about the surrounding world.

This is all possible because of languages. As he writes we can produce an infinite number of sentences, that means words will have different meanings based on the context used. So rather than having a fixed definition for each word, we’ll be defining them based on their context.

That’s the reason in programming we don’t use WordNet to find out the relationship between different words. WordNet is a library where you can get a collection of synonym sets for each word. Also, it contains hypernyms (‘is a’) relationships. It misses the nuance, for example, the word ‘good’ has a lot of synonym sets but the context is missing. One more problem with WordNet is that it requires human labor to create and update.

We need a way where we can compare the word similarity. To compare it we need to represent the words in the form of vectors. We can represent them in the form of discrete symbols. Representing them as one-hot vectors like:

hotel = [0 0 0 0 0 0 0 1 0 0 0 0 0 0 0]


So, the vector dimension will be equal to the total number of words in a vocabulary. Still, we can’t find the similarities between different words in this way as each word is represented differently and there is no way to compare. One way to fix that would be to use WordNet’s list of synonyms to get the similarity or what if we can add similarities in the vector itself?

The idea here is to create a dense vector for each word, chosen in such a way where it is similar to vectors of words that appear in similar context. Word2Vec is what we are going to use to create such a dense vector.


Let’s say we are reading a book and our task is to learn how different words are related to each other. What we will do here is we’ll take a paragraph and we will go through each word. One word at a time, let’s call it a center word. Then we have nearby words, let’s say there are five words in one sentence and center word is the middle one then two words on the left side and two on the right are nearby words. These are also known as context words. Here we take two because we have a window of size two, it can be changed.

Here we will calculate the probability of a context word given a center word or vice versa. Based on this probability we will adjust the word vectors and try to maximize the probability. This way we will get word vectors through which we will know if any two words are similar.

Word2Vec was introduced in 2013 by Mikolov et al. at Google on efficient vector representations of words. This is the abstract of the paper:

We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high-quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.


Let’s go into the details. We are going to create a neural network with a single hidden layer. In input, we will pass one hot vector representing a word and in the output, we will get a probability of that word being near the center word. This is one of that neural network where the actual goal is not the output that we get, in this case, we are interested in learning from the weights of the hidden layer. Output layer is required to decrease the loss.


Figure 1: Skip-gram predicts surrounding words given the current word.

From figure 1 we can get a basic idea of different layers in the Skip-gram model. So, there were two architectures proposed: one is Skip-gram model and the other one is the Continuous Bag-of-Words model. In the case of CBOW, we predict the center word based on context words and in the case of Skip-gram, it’s opposite. Also, it’s called the bag-of-words model as the order of words in the history does not influence the projection. In this post, we will talk about Skip-gram model.

In mathematical terms, its goal is to maximize the average log probability. Let’s say if the number of training words is T then this is the representation of average log probability:





Hidden Layer


In the above neural network architecture, you can see we have three layers and in that, we already know about the input and output layer. Let’s discuss the hidden layer in above architecture.

Hidden layer in this neural network works as a look-up table. So, let’s say if we have a one-hot vector with 1000 rows as an input and the hidden layer contains 30 features which mean a matrix of 1000 rows and 30 columns. When we multiply them then the output will be the matrix row corresponding to the 1.

Also, the features in the hidden layer can be treated as hyper-parameters which you can tune it in your application. There is no restriction, so you can try different values.

The output from the hidden layer is what we call word vectors.
We update them based on the output layer’s result.

Open source projects

If you want to use some prebuilt tools to create word representation then Facebook’s fastText is one of them.
And this is Google’s implementation of Word2Vec where they used Google news data to train the model.

Web Design

How Your Web Design Affects the Value of Your Online Business

In the modern digital marketplace, almost any business you can think of will have their own website. Whether it’s the bakery you love so much in the middle of town or a global corporation like Coca-Cola or Nike. Every brand, big and small, knows how important it can be to make an impact with their online presence.

The good news? Creating a website has never been easier. Content Management Systems like WordPress and endless hosting companies ensure that any organization can design and implement a website without even the slightest bit of technical know-how. The bad news is that it takes a lot more than just a dime-a-dozen site to make your business really stand out.

If you want an online presence that’s appealing enough to attract customers, deliver conversations, and even help you sell your business one day, then you need an incredible website design. Here, we’re going to take a look at just some of the ways that your website can change the value of your online business.

An Attractive Site Prompts Engagement and Conversions

From day one, your website helps your customers and investors to decide whether they want to interact or do business with you. Think of it this way. If you were looking for a place to go for a romantic meal with your other half, and you found a restaurant that was old, run-down, and packed full of outdated furniture, would you rush to book a table? Even if the food inside of that restaurant was great, you may never get a chance to find out, because many of us still do judge a book by its cover.

The same issue applies to your website. If your site looks well-organized, professionally designed and innovative, your customers will associate all those good things with your brand. They’ll assume that you’re a professional organization capable of delivering a professional experience. Something as simple as an easy-to-navigate menu, and an uncluttered design can be enough to help your customers see how effective your business really is.

On the other hand, an unattractive site makes it look as though you haven’t bothered to invest in your business. If you aren’t willing to spend money on making your business look good, why should your customers take you seriously? Most won’t, that’s why 38% of people stop engaging with a website if they think it’s unattractive.

A Great Website Paves the Way for Customer Loyalty

Speaking of making the right impression on your target audience, today’s customers want to establish a connection with their favorite brands. Your website is your first opportunity to show your market what you’re all about, and why they should be interested in connecting with you. After all, there’s more to a business than what it sells. A brand is how you show the personality of your organization and the vision that drives you towards success.

A good website can showcase the important elements of your brand, through the colors you choose, the words you pick, and the experiences you offer. It also helps you to give credibility to your claims. For instance, if you tell your customers that you’ll always put them first, but you fail to make your navigation menu easy to use, that creates a contrast that your customers will pick up on. In fact, 67% of customers say that bad customer experience is the reason why they leave a brand.

Show your customers that you’re authentic, caring, and intuitive with your website design, and they’ll reward you with their loyalty. That instantly improves the value of your online business by giving you long-term clients, and better revenue to show off to a potential business buyer.

Effective Design Makes You Easier to Find

Great website design isn’t just about giving your online presence more aesthetic appeal. It also gives you the tools you need to implement backend solutions that earn the attention of search engines, and eventually customers. A great designer can implement state-of-the-art SEO strategies, complete with meta descriptions, keywords, and other key components, that improve your chances of ranking higher on the search results for certain phrases.

Whether you’re planning on selling your company, or you simply want to make sure that you keep customers coming back to your website, a platform with good SEO gives you the opportunity to place your content and products in front of your customers where they can interact with them best.

It’s not just implementing keywords that you’ll need to think about for SEO purposes either! Bear in mind that bounce rate, or the length of time that people spend on your website before they hit the back button, can have an impact too. If someone comes to your website and thinks it looks unprofessional, or worse – shady, they’ll leave as soon as your pages start to load. On the other hand, if your site looks legitimate, you’re more likely to keep people around for a longer period.

A Good Design Gives People a Reason to Stick Around

Finally, the longer someone spends on your website, the more chance they have of making a purchase. A great website design convinces people to stay around for longer so that they can explore everything you have to offer. This can even lead to an increased rate of newsletter signups, conversions, and followers on your social media page too.

Studies show that when given fifteen minutes to consume content, around two-thirds of all customers would prefer to read something that looks attractive than something that looks plain. As human beings, we’re actively drawn to things that we like the look of. That means that whether you’re trying to connect with your customers or convince someone to buy your business, it all starts with having the right look.

With new innovative design factors like animations, gifs, and videos, you can even add new content elements to your website that help to build up the attractive nature of your company. While there’s more to website design than captivating your audience with shiny objects and great pictures, it helps to give them a reason to stick around.

The Bottom Line on the Value of Website Design

Ultimately, website design is the ultimate chance for you to make a powerful first impression on your audience. If you overlook the importance of a great website impact, then you’re going to end up with a serious problem, as a poorly-designed site can send your customers not only running the opposite way but also fleeing into the arms of your competitors.

At the same time, a great website design doesn’t just improve your business value by giving you more satisfied customers, it could also give you the resources you need to demand a premium when the time comes to sell your company. If your buyer can see that your well-designed website has helped you to develop a strong brand, a good reputation, and a loyal following of customers, they’ll be far more likely to pay more.

This post is written by Jock Purtle.

About the author: Jock is the founder of, and an industry leader when it comes to valuing eCommerce brands, websites, and high-growth internet companies. Over the years, he’s shared his thoughts on major publications like CNBC, Forbes, Business Insider, and many more.

Push Notifications

7 Super Useful Tips for Writing Effective Push Notifications

Push notifications are simply annoying. They interrupt you and most of the time you feel disappointed once you click on them. Even if you do not, they mostly just make you swipe to get rid of them and that is it.

However, they are necessary. How else would you know what is the weather like today or that a new episode of your favorite TV show was just released?

If you are a brand wondering how to make these interesting for your readers and effective enough to make them click it, here is a list of seven useful tips on how to make them shine.

Be witty

Think about it, witty would spark your interest too. Most people are bored with the plain old statement that says what caused the notification – they want to see something attention-grabbing, something that will make them smile.

Use humor instead of taking your brand too seriously. To the people that receive these notifications, you will seem like a friend, you will make them feel like you see the world they do. And nothing connects people as humor does. Draw inspiration from brands and celebrities whose humor you find appealing.

Create your brand voice and stick to it

Authenticity is what will drive people to you, what will make them recognize your brand whenever they see a notification. Look up to some of the most popular brands and how they do their push notifications.

This approach could potentially win many hearts – you could choose to be funny, honest, serious, sarcastic or anything else that you want to be. Just make sure that you are consistent.

Be Honest

Honesty is always the best policy. People are tired of brands overselling their product and treating them as if they do not know what is going on. This is why being honest will win you many hearts.

“For example, if your terms and services are long and boring to read, tell your users and possibly offer them a short list of what they are accepting. This transparency will also bring an element of surprise – people are so used to being lied to that a transparent brand will automatically get them to pay attention and notice your notifications”, – says Mary Pablo, a Marketing Manager at Assignment help.

Offer valuable information

If your brand is offering something new that could interest your users or there is a discount, let them know. Most people do not care about random notifications about updates and such, but they will genuinely like your effort to give them value.

Try to limit your push notifications to the ones that really offer a helping hand or useful information instead of sending out too many notifications that are ignored.

Write short and accurate notifications

Push notifications are most definitely not the place to put a lengthy text. Keep it short, sweet and catchy. Also, make sure that your notifications are accurate – no one will take you seriously with too many grammar and spelling mistakes.

Here is a list of free resources to help you write perfect notifications:

State of Writing / Viawriting – in case you need writing guides for your push notifications these websites will give you excellent tips on how to write them.
UKWritings is an online copywriting tool – recommended here – that will give you excellent tips on writing notifications that demand attention.
Academadvisor is a formatting guide, which will help you make your notifications properly formatted and structured.
Essayroo – an online proofreading tool to avoid any mistakes in the notification text
MyWritingWay and WritingPopulist are great choices and guides for the editing job
Boomessays is a title or content generating tool, which will provide you with assistance when you are a bit low on inspiration.

Stay up-to-date

This could be extremely valuable for notification writing. Take popular events like elections or football games, TV shows or popular references, memes and such to relate to your readers.

If you do this, your notifications will not be considered boring at all – in fact, they will make the readers feel connected to your brand as someone who loves the same things they do.

Evoke powerful emotions

Your notification may be about a new perfume or something similar but you can certainly connect it with more powerful things – feelings. Whether it’s nostalgia, love or happiness, your readers will like it.

Regular boring notifications do not really have what it takes to invite them in – but if you relate your notification to their feelings, they will gladly click. And the best thing is, they will look forward to your next notification.

Final Thoughts

Push notifications require short text – but it does not mean that a short text cannot pack a punch. Just make sure that whatever you do, you always think of your receivers first and what will appeal to their interest the most.


This post is written by Freddie Tubbs.

About The Author:

Freddie Tubbs is a digital analyst at Paper fellows. He also works as a marketing paper writer at Academized and Australianhelp.