Skip to main content

Typical Document Processing Operations

First rule of document pre-processing : Improper pre-processing schemes may lead to losss of lexical content.
Hence, pre-processings steps are unique to a problem. Having said that there are few pre-processing steps which applies to most of the application at hand.
They are : a. tokenization b. normalization  c. substitution.
Other well known pre-processing steps : a. Case folding b. Stemming c. Lemmatization d. Remove misspellings e. Punctuations.

What is a stemming operation ?
- Process of reducing a inflected word to its root. Where inflected word is a word with extra letter or letters added to nouns,verbs and adjectives in different grammatical forms.

What is lemmatization ?
- Here also there is reduction in inflected word to its root, however stemming resultant need not to be a proper word in vocabulary but in case of lemmatization word has be part of the given language vocabulary.

What is case folding and its usage ?
- Case-folding is a part of the Unicode standard that allows any two strings that differ from one another only by case to map to the same "case-folded" form, even when those strings include characters with complex case-mappings.[ convert all letters to a single case , either upper case of lower case whichever is chosen. ]. Helps in normalization and making text searches relavent.

Comments

Post a Comment

Popular posts from this blog

Procedure to enable wireless LAN adapter in vmware!

I have went to find the way in which wlan in Kali Linux could be made listed my airmon-ng command. Most of the videos and posts suggested me that it is necessary to have another wi-fi adapter to have connected to wifi network. But this is not necessary. The procedure to do this is as follows: 1. Download this package in Virtual Machine(Kali linux) " http://wireless.kernel.org/download/compat-wireless-2.6/compat-wireless-2010-06-26-p.tar.bz2" 2.Unzip this package in terminal i.e navigate to place where you have downloaded this file , say desktop then type : cd Desktop for unzip type : tar -jxvf compat-wireless-2010-06-26-p.tar.bz2 3. Navigate into the uncompressed directory and type: i."make unload"  (only words within the quotes are the commands) ii."make load" Done!  now airmon-ng  should have the list of the required devices.
2 comments
Read more

"Swift" from Programming Template

About Memory: 1. How do I get memory, i.e how are variables declared. Any specific facts and constrains about variables ? If type annotation like var Varb:Float ; is not used then it is mandatory to assign the initial value to the variable. "Var" keyword should get you the required memory. 2. How do we get the entry point for the program execution ? [ Example : main() in the C program ] Script based. There is no particular entry point. 3. How does dynamic memory handled ? Is it supported in first place ? Instances for the classes can be created. Automatic Reference counting mechanism has the strong reference to one of the variable. Till there is strong reference to the variable it'll be there in the memory. Once there is no strong reference it'll be de-initialized. [ Ref : https://developer.apple.com/library/content/documentation/Swift/Conceptual/Swift_Programming_Language/AutomaticReferenceCounting.html ] 4. What are the datatypes which are supported an...
Post a Comment
Read more

Create One-Click Shutdown and Reboot Shortcuts

First, create a shortcut on your desktop by right-clicking on the desktop, choosing New, and then choosing Shortcut. The Create Shortcut Wizard appears. In the box asking for the location of the shortcut, type shutdown. After you create the shortcut, double-clicking on it will shut down your PC. But you can do much more with a shutdown shortcut than merely shut down your PC. You can add any combination of several switches to do extra duty, like this: shutdown -r -t 01 -c "Rebooting your PC" Double-clicking on that shortcut will reboot your PC after a one-second delay and display the message "Rebooting your PC." The shutdown command includes a variety of switches you can use to customize it. Table 1-3 lists all of them and describes their use. I use this technique to create two shutdown shortcuts on my desktop—one for turning off my PC, and one for rebooting. Here are the ones I use: shutdown -s -t 03 -c "Bye Bye m8!" shutdown -r -t 03 -c ...
Post a Comment
Read more